[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:31:42 dist-test-slave-4m58 krb5kdc[4436](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:31:42 dist-test-slave-4m58 krb5kdc[4436](info): set up 2 sockets
Jun 19 20:31:42 dist-test-slave-4m58 krb5kdc[4436](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260619 20:31:42.997018 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.966s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:31:43 dist-test-slave-4m58 krb5kdc[4436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:31:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:31:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
I20260619 20:31:43.123518 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:37717
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:45155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:37717
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:31:43.209851 4449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:31:43.210085 4449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:31:43.210110 4449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:31:43.211385 4449 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:31:43.211439 4449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:31:43.211452 4449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:31:43.211463 4449 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:31:43.211474 4449 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:31:43.213089 4449 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:45155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:37717
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:37717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:31:43.213335 4449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:31:43.213560 4449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:31:43.219115 4454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:43.219100 4455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:43.219130 4449 server_base.cc:1034] running on GCE node
W20260619 20:31:43.219106 4457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:43.219641 4449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:31:43.220002 4449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:31:43.221174 4449 hybrid_clock.cc:648] HybridClock initialized: now 1781901103221160 us; error 31 us; skew 500 ppm
Jun 19 20:31:43 dist-test-slave-4m58 krb5kdc[4436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901103, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:31:43.223932 4449 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:31:43.224522 4449 webserver.cc:460] Webserver started at http://127.4.83.126:37739/ using document root <none> and password file <none>
I20260619 20:31:43.224764 4449 fs_manager.cc:359] Metadata directory not provided
I20260619 20:31:43.224815 4449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:31:43.224954 4449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:31:43.226076 4449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "83707e090681471badfc34a09b762535"
format_stamp: "Formatted at 2026-06-19 20:31:43 on dist-test-slave-4m58"
server_key: "eb68ba17ea67ef3fb3eb6d4ab84fd904"
server_key_iv: "07f23a2feb63b4ba849585b554c6bc65"
server_key_version: "encryptionkey@0"
I20260619 20:31:43.226428 4449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "83707e090681471badfc34a09b762535"
format_stamp: "Formatted at 2026-06-19 20:31:43 on dist-test-slave-4m58"
server_key: "eb68ba17ea67ef3fb3eb6d4ab84fd904"
server_key_iv: "07f23a2feb63b4ba849585b554c6bc65"
server_key_version: "encryptionkey@0"
I20260619 20:31:43.227718 4449 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 20:31:43.228585 4464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:43.228741 4449 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:31:43.228817 4449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "83707e090681471badfc34a09b762535"
format_stamp: "Formatted at 2026-06-19 20:31:43 on dist-test-slave-4m58"
server_key: "eb68ba17ea67ef3fb3eb6d4ab84fd904"
server_key_iv: "07f23a2feb63b4ba849585b554c6bc65"
server_key_version: "encryptionkey@0"
I20260619 20:31:43.228883 4449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:31:43.239460 4449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:31:43.241878 4449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:31:43.242027 4449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:31:43.246031 4449 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:37717
I20260619 20:31:43.246420 4449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:31:43.246466 4516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:37717 every 8 connection(s)
I20260619 20:31:43.247059 4517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:43.248790 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 4449
I20260619 20:31:43.248883 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:31:43.249446 4517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83707e090681471badfc34a09b762535: Bootstrap starting.
I20260619 20:31:43.249514 4429 external_mini_cluster.cc:1428] Setting key c142903dc04dc51599c147609265f32e
I20260619 20:31:43.250046 4517 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83707e090681471badfc34a09b762535: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:43.250363 4517 log.cc:824] T 00000000000000000000000000000000 P 83707e090681471badfc34a09b762535: Log is configured to *not* fsync() on all Append() calls
I20260619 20:31:43.251130 4517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83707e090681471badfc34a09b762535: No bootstrap required, opened a new log
I20260619 20:31:43.252858 4517 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83707e090681471badfc34a09b762535 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83707e090681471badfc34a09b762535" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37717 } }
I20260619 20:31:43.253036 4517 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83707e090681471badfc34a09b762535 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:43.253072 4517 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83707e090681471badfc34a09b762535 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83707e090681471badfc34a09b762535, State: Initialized, Role: FOLLOWER
I20260619 20:31:43.253206 4517 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83707e090681471badfc34a09b762535 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83707e090681471badfc34a09b762535" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37717 } }
I20260619 20:31:43.253276 4517 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83707e090681471badfc34a09b762535 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:31:43.253305 4517 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83707e090681471badfc34a09b762535 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:31:43.253353 4517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83707e090681471badfc34a09b762535 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:43.254267 4517 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83707e090681471badfc34a09b762535 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83707e090681471badfc34a09b762535" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37717 } }
I20260619 20:31:43.254366 4517 leader_election.cc:302] T 00000000000000000000000000000000 P 83707e090681471badfc34a09b762535 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83707e090681471badfc34a09b762535; no voters:
I20260619 20:31:43.254525 4517 leader_election.cc:288] T 00000000000000000000000000000000 P 83707e090681471badfc34a09b762535 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:31:43.254561 4522 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83707e090681471badfc34a09b762535 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:31:43.254684 4522 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83707e090681471badfc34a09b762535 [term 1 LEADER]: Becoming Leader. State: Replica: 83707e090681471badfc34a09b762535, State: Running, Role: LEADER
I20260619 20:31:43.254797 4517 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83707e090681471badfc34a09b762535 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:31:43.254762 4522 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83707e090681471badfc34a09b762535 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83707e090681471badfc34a09b762535" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37717 } }
I20260619 20:31:43.255422 4524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83707e090681471badfc34a09b762535 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83707e090681471badfc34a09b762535" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83707e090681471badfc34a09b762535" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37717 } } }
I20260619 20:31:43.255585 4524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83707e090681471badfc34a09b762535 [sys.catalog]: This master's current role is: LEADER
I20260619 20:31:43.256117 4523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83707e090681471badfc34a09b762535 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83707e090681471badfc34a09b762535. Latest consensus state: current_term: 1 leader_uuid: "83707e090681471badfc34a09b762535" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83707e090681471badfc34a09b762535" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37717 } } }
I20260619 20:31:43.256299 4531 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:31:43.256394 4523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83707e090681471badfc34a09b762535 [sys.catalog]: This master's current role is: LEADER
I20260619 20:31:43.256778 4531 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:31:43.258154 4531 catalog_manager.cc:1347] Generated new cluster ID: 175ebd57e30f4777abe734eb9ad71fb9
I20260619 20:31:43.258236 4531 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:31:43 dist-test-slave-4m58 krb5kdc[4436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:31:43.266330 4519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:43.250349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60678 (local address 127.4.83.126:37717)
0619 20:31:43.250652 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:43.250661 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:43.250844 (+ 183us) server_negotiation.cc:407] Connection header received
0619 20:31:43.252827 (+ 1983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:43.252835 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:43.252894 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:43.253032 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:43.254711 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:43.256086 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:43.257544 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:43.258009 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:43.258123 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:43.261878 (+ 3755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:43.261883 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:43.261884 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:43.261895 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:43.264222 (+ 2327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:43.264671 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:43.264674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:43.264675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:43.264790 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:43.265060 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:43.265061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:43.265062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:43.265345 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:43.265493 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:43.265766 (+ 273us) server_negotiation.cc:299] Negotiation successful
0619 20:31:43.265905 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":257,"thread_start_us":79,"threads_started":1}
I20260619 20:31:43.271485 4531 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:31:43.272224 4531 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:31:43.289669 4531 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83707e090681471badfc34a09b762535: Generated new TSK 0
I20260619 20:31:43.289901 4531 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.83.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
I20260619 20:31:43.327272 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:0
--local_ip_for_outbound_sockets=127.4.83.65
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:37717
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:45155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:31:43.430960 4543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:31:43.431174 4543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:31:43.431200 4543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:31:43.432466 4543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:31:43.432510 4543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:31:43.432538 4543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:31:43.434060 4543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:45155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:37717
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:31:43.434262 4543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:31:43.434435 4543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:31:43.439275 4549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:43.439325 4548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:43.439378 4551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:43.439458 4543 server_base.cc:1034] running on GCE node
I20260619 20:31:43.439718 4543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:31:43.439973 4543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:31:43.441135 4543 hybrid_clock.cc:648] HybridClock initialized: now 1781901103441128 us; error 29 us; skew 500 ppm
Jun 19 20:31:43 dist-test-slave-4m58 krb5kdc[4436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901103, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:31:43.443611 4543 init.cc:377] Logged in from keytab as kudu/127.4.83.65@KRBTEST.COM (short username kudu)
I20260619 20:31:43.444098 4543 webserver.cc:460] Webserver started at http://127.4.83.65:44045/ using document root <none> and password file <none>
I20260619 20:31:43.444298 4543 fs_manager.cc:359] Metadata directory not provided
I20260619 20:31:43.444345 4543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:31:43.444473 4543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:31:43.445528 4543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-0/data/instance:
uuid: "994e1ae4a997402d9818460bab437442"
format_stamp: "Formatted at 2026-06-19 20:31:43 on dist-test-slave-4m58"
server_key: "bb8296bee7a3c8028a899182f9f0d057"
server_key_iv: "0eafcaad66e2428af0f4533d1c9ecbdb"
server_key_version: "encryptionkey@0"
I20260619 20:31:43.445850 4543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance:
uuid: "994e1ae4a997402d9818460bab437442"
format_stamp: "Formatted at 2026-06-19 20:31:43 on dist-test-slave-4m58"
server_key: "bb8296bee7a3c8028a899182f9f0d057"
server_key_iv: "0eafcaad66e2428af0f4533d1c9ecbdb"
server_key_version: "encryptionkey@0"
I20260619 20:31:43.447122 4543 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:31:43.447880 4558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:43.448066 4543 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:31:43.448148 4543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "994e1ae4a997402d9818460bab437442"
format_stamp: "Formatted at 2026-06-19 20:31:43 on dist-test-slave-4m58"
server_key: "bb8296bee7a3c8028a899182f9f0d057"
server_key_iv: "0eafcaad66e2428af0f4533d1c9ecbdb"
server_key_version: "encryptionkey@0"
I20260619 20:31:43.448217 4543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:31:43.466420 4543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:31:43.468866 4543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:31:43.469038 4543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:31:43.469287 4543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:31:43.469609 4543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:31:43.469645 4543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:43.469676 4543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:31:43.469692 4543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:43.475636 4543 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:37573
I20260619 20:31:43.475719 4671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:37573 every 8 connection(s)
I20260619 20:31:43.476042 4543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
Jun 19 20:31:43 dist-test-slave-4m58 krb5kdc[4436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901103, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:31:43.483209 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 4543
I20260619 20:31:43.483310 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance
I20260619 20:31:43.483465 4429 external_mini_cluster.cc:1428] Setting key 91a8bc94cd89e228a0a3bba8d3dafa7d
I20260619 20:31:43.488127 4519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:43.476834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:39377 (local address 127.4.83.126:37717)
0619 20:31:43.476905 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:43.476908 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:43.477136 (+ 228us) server_negotiation.cc:407] Connection header received
0619 20:31:43.477660 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:43.477661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:43.477674 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:43.477723 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:43.478835 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:43.479348 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:43.480423 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:43.480638 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:43.480696 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:43.483707 (+ 3011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:43.483709 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:43.483710 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:43.483719 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:43.485615 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:43.486353 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:43.486355 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:43.486355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:43.486438 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:43.486774 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:43.486775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:43.486775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:43.486979 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:43.487056 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:43.487945 (+ 889us) server_negotiation.cc:299] Negotiation successful
0619 20:31:43.488050 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 20:31:43.488497 4674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:43.476807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:37717 (local address 127.4.83.65:39377)
0619 20:31:43.477018 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:43.477055 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:43.477561 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:43.477804 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:43.477808 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:43.478519 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:31:43.478731 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:43.478736 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:43.479442 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:43.479444 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:43.480329 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:43.480332 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:43.480712 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:43.480712 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:43.480833 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:43.481404 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:31:43.481413 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:31:43.483580 (+ 2167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:31:43.485713 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:43.485725 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:43.485728 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:43.486101 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:43.486553 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:43.486554 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:43.486555 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:43.486702 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:43.487180 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:31:43.487186 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:31:43.487641 (+ 455us) client_negotiation.cc:769] Sending connection context
0619 20:31:43.487891 (+ 250us) client_negotiation.cc:240] Negotiation successful
0619 20:31:43.488187 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":88,"threads_started":1}
I20260619 20:31:43.489225 4672 heartbeater.cc:347] Connected to a master server at 127.4.83.126:37717
I20260619 20:31:43.489346 4672 heartbeater.cc:464] Registering TS with master...
I20260619 20:31:43.489554 4672 heartbeater.cc:511] Master 127.4.83.126:37717 requested a full tablet report, sending...
I20260619 20:31:43.490094 4481 ts_manager.cc:194] Registered new tserver with Master: 994e1ae4a997402d9818460bab437442 (127.4.83.65:37573)
I20260619 20:31:43.490710 4481 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.65@KRBTEST.COM'} at 127.4.83.65:39377
WARNING: no policy specified for kudu/127.4.83.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
I20260619 20:31:43.513422 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:0
--local_ip_for_outbound_sockets=127.4.83.66
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:37717
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:45155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:31:43.597739 4677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:31:43.598114 4677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:31:43.598158 4677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:31:43.599472 4677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:31:43.599531 4677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:31:43.599562 4677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:31:43.601121 4677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:45155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:37717
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:31:43.601333 4677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:31:43.601527 4677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:31:43.606535 4677 server_base.cc:1034] running on GCE node
W20260619 20:31:43.606518 4683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:43.606447 4682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:43.606482 4685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:43.606865 4677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:31:43.607105 4677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:31:43.608246 4677 hybrid_clock.cc:648] HybridClock initialized: now 1781901103608237 us; error 33 us; skew 500 ppm
Jun 19 20:31:43 dist-test-slave-4m58 krb5kdc[4436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901103, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:31:43.610687 4677 init.cc:377] Logged in from keytab as kudu/127.4.83.66@KRBTEST.COM (short username kudu)
I20260619 20:31:43.611117 4677 webserver.cc:460] Webserver started at http://127.4.83.66:42919/ using document root <none> and password file <none>
I20260619 20:31:43.611320 4677 fs_manager.cc:359] Metadata directory not provided
I20260619 20:31:43.611364 4677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:31:43.611483 4677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:31:43.612704 4677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-1/data/instance:
uuid: "2d117978f35c461d81ae16e86522ff3d"
format_stamp: "Formatted at 2026-06-19 20:31:43 on dist-test-slave-4m58"
server_key: "45c2631d937be02c0cb098b5acd597fb"
server_key_iv: "62604395c8533adf4cb6cf602cf632c9"
server_key_version: "encryptionkey@0"
I20260619 20:31:43.613058 4677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance:
uuid: "2d117978f35c461d81ae16e86522ff3d"
format_stamp: "Formatted at 2026-06-19 20:31:43 on dist-test-slave-4m58"
server_key: "45c2631d937be02c0cb098b5acd597fb"
server_key_iv: "62604395c8533adf4cb6cf602cf632c9"
server_key_version: "encryptionkey@0"
I20260619 20:31:43.614507 4677 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:31:43.615406 4692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:43.615630 4677 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:31:43.615726 4677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "2d117978f35c461d81ae16e86522ff3d"
format_stamp: "Formatted at 2026-06-19 20:31:43 on dist-test-slave-4m58"
server_key: "45c2631d937be02c0cb098b5acd597fb"
server_key_iv: "62604395c8533adf4cb6cf602cf632c9"
server_key_version: "encryptionkey@0"
I20260619 20:31:43.615789 4677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:31:43.640933 4677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:31:43.643101 4677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:31:43.643253 4677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:31:43.643507 4677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:31:43.643901 4677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:31:43.643941 4677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:43.643973 4677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:31:43.643996 4677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:43.649595 4677 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:39153
I20260619 20:31:43.649708 4805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:39153 every 8 connection(s)
I20260619 20:31:43.650501 4677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
Jun 19 20:31:43 dist-test-slave-4m58 krb5kdc[4436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901103, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:31:43.659015 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 4677
I20260619 20:31:43.659108 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance
I20260619 20:31:43.659265 4429 external_mini_cluster.cc:1428] Setting key 6fe84937b951ca06269ab29f86ffbdd1
I20260619 20:31:43.660436 4519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:43.651298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:34289 (local address 127.4.83.126:37717)
0619 20:31:43.651380 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:43.651382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:43.651560 (+ 178us) server_negotiation.cc:407] Connection header received
0619 20:31:43.652129 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:43.652131 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:43.652142 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:43.652199 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:43.653153 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:43.653681 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:43.654831 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:43.655071 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:43.655136 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:43.657427 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:43.657429 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:43.657429 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:43.657437 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:43.658786 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:43.659279 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:43.659281 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:43.659281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:43.659354 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:43.659651 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:43.659651 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:43.659651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:43.659770 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:43.659857 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:43.660155 (+ 298us) server_negotiation.cc:299] Negotiation successful
0619 20:31:43.660339 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 20:31:43.660651 4808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:43.651236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:37717 (local address 127.4.83.66:34289)
0619 20:31:43.651391 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:43.651432 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:43.652034 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:43.652260 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:43.652264 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:43.652846 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:31:43.653051 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:43.653056 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:43.653804 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:43.653805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:43.654716 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:43.654719 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:43.655152 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:43.655153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:43.655280 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:43.655791 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:31:43.655799 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:31:43.657324 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:31:43.658890 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:43.658898 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:43.658900 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:43.659181 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:43.659444 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:43.659446 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:43.659446 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:43.659587 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:43.659857 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:31:43.659864 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:31:43.660019 (+ 155us) client_negotiation.cc:769] Sending connection context
0619 20:31:43.660119 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 20:31:43.660423 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
I20260619 20:31:43.661759 4806 heartbeater.cc:347] Connected to a master server at 127.4.83.126:37717
I20260619 20:31:43.661859 4806 heartbeater.cc:464] Registering TS with master...
I20260619 20:31:43.662042 4806 heartbeater.cc:511] Master 127.4.83.126:37717 requested a full tablet report, sending...
I20260619 20:31:43.662408 4481 ts_manager.cc:194] Registered new tserver with Master: 2d117978f35c461d81ae16e86522ff3d (127.4.83.66:39153)
I20260619 20:31:43.662884 4481 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.66@KRBTEST.COM'} at 127.4.83.66:34289
WARNING: no policy specified for kudu/127.4.83.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
I20260619 20:31:43.687664 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:0
--local_ip_for_outbound_sockets=127.4.83.67
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:37717
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:45155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:31:43.769865 4811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:31:43.770087 4811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:31:43.770119 4811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:31:43.771281 4811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:31:43.771328 4811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:31:43.771364 4811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:31:43.772895 4811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:45155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:37717
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:31:43.773149 4811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:31:43.773367 4811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:31:43.778360 4819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:43.778396 4816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:43.778414 4811 server_base.cc:1034] running on GCE node
W20260619 20:31:43.778380 4817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:43.778800 4811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:31:43.779089 4811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:31:43.780258 4811 hybrid_clock.cc:648] HybridClock initialized: now 1781901103780220 us; error 56 us; skew 500 ppm
Jun 19 20:31:43 dist-test-slave-4m58 krb5kdc[4436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901103, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:31:43.782886 4811 init.cc:377] Logged in from keytab as kudu/127.4.83.67@KRBTEST.COM (short username kudu)
I20260619 20:31:43.783320 4811 webserver.cc:460] Webserver started at http://127.4.83.67:37435/ using document root <none> and password file <none>
I20260619 20:31:43.783541 4811 fs_manager.cc:359] Metadata directory not provided
I20260619 20:31:43.783594 4811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:31:43.783720 4811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:31:43.784829 4811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-2/data/instance:
uuid: "0597d9e542ec47ef9e97cec970c5315f"
format_stamp: "Formatted at 2026-06-19 20:31:43 on dist-test-slave-4m58"
server_key: "4ec4a50262592ad50d6833f459061034"
server_key_iv: "2dabcf73b8c853ef758b3e0269a02685"
server_key_version: "encryptionkey@0"
I20260619 20:31:43.785156 4811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance:
uuid: "0597d9e542ec47ef9e97cec970c5315f"
format_stamp: "Formatted at 2026-06-19 20:31:43 on dist-test-slave-4m58"
server_key: "4ec4a50262592ad50d6833f459061034"
server_key_iv: "2dabcf73b8c853ef758b3e0269a02685"
server_key_version: "encryptionkey@0"
I20260619 20:31:43.786475 4811 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:31:43.787238 4826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:43.787432 4811 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:31:43.787521 4811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "0597d9e542ec47ef9e97cec970c5315f"
format_stamp: "Formatted at 2026-06-19 20:31:43 on dist-test-slave-4m58"
server_key: "4ec4a50262592ad50d6833f459061034"
server_key_iv: "2dabcf73b8c853ef758b3e0269a02685"
server_key_version: "encryptionkey@0"
I20260619 20:31:43.787588 4811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:31:43.799814 4811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:31:43.802260 4811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:31:43.802395 4811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:31:43.802603 4811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:31:43.802909 4811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:31:43.802945 4811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:43.802976 4811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:31:43.803001 4811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:43.808822 4811 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:38277
I20260619 20:31:43.808923 4939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:38277 every 8 connection(s)
I20260619 20:31:43.809209 4811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
I20260619 20:31:43.812244 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 4811
I20260619 20:31:43.812325 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance
I20260619 20:31:43.812460 4429 external_mini_cluster.cc:1428] Setting key 64ee8f28487300ff274219de732c3a1e
Jun 19 20:31:43 dist-test-slave-4m58 krb5kdc[4436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901103, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:31:43.819386 4519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:43.810000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:40397 (local address 127.4.83.126:37717)
0619 20:31:43.810096 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:43.810098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:43.810267 (+ 169us) server_negotiation.cc:407] Connection header received
0619 20:31:43.810740 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:43.810741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:43.810753 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:43.810849 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:43.811805 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:43.812415 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:43.813784 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:43.814008 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:43.814057 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:43.816311 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:43.816312 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:43.816312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:43.816320 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:43.817782 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:43.818290 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:43.818291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:43.818291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:43.818353 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:43.818648 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:43.818649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:43.818649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:43.818771 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:43.818826 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:43.819218 (+ 392us) server_negotiation.cc:299] Negotiation successful
0619 20:31:43.819299 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 20:31:43.819624 4942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:43.809959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:37717 (local address 127.4.83.67:40397)
0619 20:31:43.810114 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:43.810158 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:43.810633 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:43.810868 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:43.810871 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:43.811488 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:31:43.811695 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:43.811700 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:43.812531 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:43.812532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:43.813700 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:43.813703 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:43.814066 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:43.814067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:43.814168 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:43.814620 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:31:43.814627 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:31:43.816207 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:31:43.817911 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:43.817920 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:43.817922 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:43.818182 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:43.818450 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:43.818451 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:43.818451 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:43.818589 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:43.818851 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:31:43.818858 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:31:43.819037 (+ 179us) client_negotiation.cc:769] Sending connection context
0619 20:31:43.819140 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 20:31:43.819373 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":84,"threads_started":1}
I20260619 20:31:43.820200 4940 heartbeater.cc:347] Connected to a master server at 127.4.83.126:37717
I20260619 20:31:43.820292 4940 heartbeater.cc:464] Registering TS with master...
I20260619 20:31:43.820470 4940 heartbeater.cc:511] Master 127.4.83.126:37717 requested a full tablet report, sending...
I20260619 20:31:43.821180 4481 ts_manager.cc:194] Registered new tserver with Master: 0597d9e542ec47ef9e97cec970c5315f (127.4.83.67:38277)
I20260619 20:31:43.821586 4481 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.67@KRBTEST.COM'} at 127.4.83.67:40397
I20260619 20:31:43.824544 4429 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 20:31:43 dist-test-slave-4m58 krb5kdc[4436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901103, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 20:31:43 dist-test-slave-4m58 krb5kdc[4436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901103, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.83.65@KRBTEST.COM
I20260619 20:31:43.847167 4947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:43.839691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51826 (local address 127.4.83.65:37573)
0619 20:31:43.839887 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:43.839892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:43.839902 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:31:43.839927 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:43.839929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:43.839942 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:43.840009 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:43.840588 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:43.841312 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:43.842036 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:43.842372 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:43.842433 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:43.844745 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:43.844754 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:43.844755 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:43.844762 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:43.846115 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:43.846435 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:43.846438 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:43.846438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:43.846500 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:43.846642 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:43.846642 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:43.846643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:43.846774 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:43.846886 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:43.847009 (+ 123us) server_negotiation.cc:299] Negotiation successful
0619 20:31:43.847080 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":66,"threads_started":1}
W20260619 20:31:43.847576 4586 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51826
Jun 19 20:31:43 dist-test-slave-4m58 krb5kdc[4436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 20:31:43 dist-test-slave-4m58 krb5kdc[4436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.65@KRBTEST.COM
I20260619 20:31:43.869567 4947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:43.862608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51838 (local address 127.4.83.65:37573)
0619 20:31:43.862689 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:43.862690 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:43.862730 (+ 40us) server_negotiation.cc:407] Connection header received
0619 20:31:43.862794 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:43.862795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:43.862806 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:43.862855 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:43.863430 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:43.863982 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:43.864677 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:43.864895 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:43.864958 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:43.867154 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:43.867156 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:43.867156 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:43.867163 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:43.868491 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:43.868815 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:43.868815 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:43.868816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:43.868861 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:43.869085 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:43.869086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:43.869087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:43.869196 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:43.869304 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:43.869423 (+ 119us) server_negotiation.cc:299] Negotiation successful
0619 20:31:43.869495 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 20:31:43.870226 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 4543
I20260619 20:31:43.874020 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 4677
I20260619 20:31:43.878202 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 4811
I20260619 20:31:43.882265 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 4449
2026-06-19T20:31:43Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1876 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:31:43 dist-test-slave-4m58 krb5kdc[4954](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:31:43 dist-test-slave-4m58 krb5kdc[4954](info): set up 2 sockets
Jun 19 20:31:43 dist-test-slave-4m58 krb5kdc[4954](info): commencing operation
krb5kdc: starting...
W20260619 20:31:44.844218 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.943s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:31:44 dist-test-slave-4m58 krb5kdc[4954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:31:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:31:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
I20260619 20:31:44.967135 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:37747
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:36883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:37747
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:31:45.052596 4967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:31:45.052831 4967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:31:45.052875 4967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:31:45.054162 4967 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:31:45.054219 4967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:31:45.054374 4967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:31:45.054430 4967 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:31:45.054456 4967 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:31:45.056347 4967 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:36883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:37747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:37747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:31:45.056654 4967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:31:45.057014 4967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:31:45.062404 4967 server_base.cc:1034] running on GCE node
W20260619 20:31:45.062414 4973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:45.062414 4972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:45.062551 4975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:45.062858 4967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:31:45.063180 4967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:31:45.064352 4967 hybrid_clock.cc:648] HybridClock initialized: now 1781901105064313 us; error 52 us; skew 500 ppm
Jun 19 20:31:45 dist-test-slave-4m58 krb5kdc[4954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901105, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:31:45.066998 4967 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:31:45.067533 4967 webserver.cc:460] Webserver started at http://127.4.83.126:33081/ using document root <none> and password file <none>
I20260619 20:31:45.067749 4967 fs_manager.cc:359] Metadata directory not provided
I20260619 20:31:45.067930 4967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:31:45.068084 4967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:31:45.069228 4967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "589f3d97efa347e8a1aa5adc0d4d5250"
format_stamp: "Formatted at 2026-06-19 20:31:45 on dist-test-slave-4m58"
server_key: "9830dd5a21fe516108b3945ca70a8a1b"
server_key_iv: "2cff58da5dfae95ffb8e083aefdf0a5e"
server_key_version: "encryptionkey@0"
I20260619 20:31:45.069640 4967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "589f3d97efa347e8a1aa5adc0d4d5250"
format_stamp: "Formatted at 2026-06-19 20:31:45 on dist-test-slave-4m58"
server_key: "9830dd5a21fe516108b3945ca70a8a1b"
server_key_iv: "2cff58da5dfae95ffb8e083aefdf0a5e"
server_key_version: "encryptionkey@0"
I20260619 20:31:45.071125 4967 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:31:45.072050 4982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:45.072353 4967 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:31:45.072449 4967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "589f3d97efa347e8a1aa5adc0d4d5250"
format_stamp: "Formatted at 2026-06-19 20:31:45 on dist-test-slave-4m58"
server_key: "9830dd5a21fe516108b3945ca70a8a1b"
server_key_iv: "2cff58da5dfae95ffb8e083aefdf0a5e"
server_key_version: "encryptionkey@0"
I20260619 20:31:45.072536 4967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:31:45.088377 4967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:31:45.090898 4967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:31:45.091061 4967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:31:45.094980 4967 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:37747
I20260619 20:31:45.095011 5034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:37747 every 8 connection(s)
I20260619 20:31:45.095371 4967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:31:45.096308 5035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:45.098666 5035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 589f3d97efa347e8a1aa5adc0d4d5250: Bootstrap starting.
I20260619 20:31:45.099232 5035 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 589f3d97efa347e8a1aa5adc0d4d5250: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:45.099537 5035 log.cc:824] T 00000000000000000000000000000000 P 589f3d97efa347e8a1aa5adc0d4d5250: Log is configured to *not* fsync() on all Append() calls
I20260619 20:31:45.100733 5035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 589f3d97efa347e8a1aa5adc0d4d5250: No bootstrap required, opened a new log
I20260619 20:31:45.102386 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 4967
I20260619 20:31:45.102505 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:31:45.102674 4429 external_mini_cluster.cc:1428] Setting key b21af7700bd47b4b2299be768d20a031
I20260619 20:31:45.103096 5035 raft_consensus.cc:348] T 00000000000000000000000000000000 P 589f3d97efa347e8a1aa5adc0d4d5250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589f3d97efa347e8a1aa5adc0d4d5250" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37747 } }
I20260619 20:31:45.103338 5035 raft_consensus.cc:374] T 00000000000000000000000000000000 P 589f3d97efa347e8a1aa5adc0d4d5250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:45.103381 5035 raft_consensus.cc:729] T 00000000000000000000000000000000 P 589f3d97efa347e8a1aa5adc0d4d5250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 589f3d97efa347e8a1aa5adc0d4d5250, State: Initialized, Role: FOLLOWER
I20260619 20:31:45.103545 5035 consensus_queue.cc:260] T 00000000000000000000000000000000 P 589f3d97efa347e8a1aa5adc0d4d5250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589f3d97efa347e8a1aa5adc0d4d5250" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37747 } }
I20260619 20:31:45.103638 5035 raft_consensus.cc:388] T 00000000000000000000000000000000 P 589f3d97efa347e8a1aa5adc0d4d5250 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:31:45.103681 5035 raft_consensus.cc:482] T 00000000000000000000000000000000 P 589f3d97efa347e8a1aa5adc0d4d5250 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:31:45.103734 5035 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 589f3d97efa347e8a1aa5adc0d4d5250 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:45.104827 5035 raft_consensus.cc:504] T 00000000000000000000000000000000 P 589f3d97efa347e8a1aa5adc0d4d5250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589f3d97efa347e8a1aa5adc0d4d5250" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37747 } }
I20260619 20:31:45.104981 5035 leader_election.cc:302] T 00000000000000000000000000000000 P 589f3d97efa347e8a1aa5adc0d4d5250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 589f3d97efa347e8a1aa5adc0d4d5250; no voters:
I20260619 20:31:45.105201 5035 leader_election.cc:288] T 00000000000000000000000000000000 P 589f3d97efa347e8a1aa5adc0d4d5250 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:31:45.105540 5035 sys_catalog.cc:564] T 00000000000000000000000000000000 P 589f3d97efa347e8a1aa5adc0d4d5250 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:31:45.105671 5040 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 589f3d97efa347e8a1aa5adc0d4d5250 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:31:45.105924 5040 raft_consensus.cc:686] T 00000000000000000000000000000000 P 589f3d97efa347e8a1aa5adc0d4d5250 [term 1 LEADER]: Becoming Leader. State: Replica: 589f3d97efa347e8a1aa5adc0d4d5250, State: Running, Role: LEADER
I20260619 20:31:45.106153 5040 consensus_queue.cc:237] T 00000000000000000000000000000000 P 589f3d97efa347e8a1aa5adc0d4d5250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589f3d97efa347e8a1aa5adc0d4d5250" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37747 } }
I20260619 20:31:45.106741 5040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 589f3d97efa347e8a1aa5adc0d4d5250 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "589f3d97efa347e8a1aa5adc0d4d5250" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589f3d97efa347e8a1aa5adc0d4d5250" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37747 } } }
I20260619 20:31:45.106878 5040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 589f3d97efa347e8a1aa5adc0d4d5250 [sys.catalog]: This master's current role is: LEADER
I20260619 20:31:45.107227 5040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 589f3d97efa347e8a1aa5adc0d4d5250 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 589f3d97efa347e8a1aa5adc0d4d5250. Latest consensus state: current_term: 1 leader_uuid: "589f3d97efa347e8a1aa5adc0d4d5250" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589f3d97efa347e8a1aa5adc0d4d5250" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37747 } } }
I20260619 20:31:45.107458 5040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 589f3d97efa347e8a1aa5adc0d4d5250 [sys.catalog]: This master's current role is: LEADER
W20260619 20:31:45.107880 5054 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 589f3d97efa347e8a1aa5adc0d4d5250: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:31:45.107945 5054 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:31:45.108130 5055 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:31:45.108317 5055 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:31:45.109858 5055 catalog_manager.cc:1347] Generated new cluster ID: 0c0227814ae5459682bb2083b201a1fe
I20260619 20:31:45.109912 5055 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:31:45 dist-test-slave-4m58 krb5kdc[4954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:31:45.119635 5055 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:31:45.120023 5039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:45.103457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48122 (local address 127.4.83.126:37747)
0619 20:31:45.103710 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:45.103724 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:45.103766 (+ 42us) server_negotiation.cc:407] Connection header received
0619 20:31:45.103944 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:45.103954 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:45.104027 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:45.104237 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:45.105676 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.107531 (+ 1855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.108860 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.109744 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.110418 (+ 674us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:45.113388 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:45.113396 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:45.113397 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:45.113416 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:45.117066 (+ 3650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:45.117861 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:45.117868 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:45.117868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:45.118116 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:45.118437 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:45.118438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:45.118439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:45.118979 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:45.119228 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:45.119337 (+ 109us) server_negotiation.cc:299] Negotiation successful
0619 20:31:45.119557 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":129,"threads_started":1}
I20260619 20:31:45.120836 5055 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:31:45.128243 5055 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 589f3d97efa347e8a1aa5adc0d4d5250: Generated new TSK 0
I20260619 20:31:45.128504 5055 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.83.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
I20260619 20:31:45.164335 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:0
--local_ip_for_outbound_sockets=127.4.83.65
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:37747
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:36883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:31:45.253098 5061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:31:45.253345 5061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:31:45.253382 5061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:31:45.254703 5061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:31:45.254781 5061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:31:45.254825 5061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:31:45.256630 5061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:36883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:37747
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:31:45.256913 5061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:31:45.257143 5061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:31:45.262667 5069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:45.262667 5066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:45.262979 5067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:45.263590 5061 server_base.cc:1034] running on GCE node
I20260619 20:31:45.263919 5061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:31:45.264179 5061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:31:45.265368 5061 hybrid_clock.cc:648] HybridClock initialized: now 1781901105265346 us; error 39 us; skew 500 ppm
Jun 19 20:31:45 dist-test-slave-4m58 krb5kdc[4954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901105, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:31:45.268244 5061 init.cc:377] Logged in from keytab as kudu/127.4.83.65@KRBTEST.COM (short username kudu)
I20260619 20:31:45.268869 5061 webserver.cc:460] Webserver started at http://127.4.83.65:39803/ using document root <none> and password file <none>
I20260619 20:31:45.269102 5061 fs_manager.cc:359] Metadata directory not provided
I20260619 20:31:45.269155 5061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:31:45.269266 5061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:31:45.270372 5061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-0/data/instance:
uuid: "b6232f5e56bc4b51b381d466fa0d5137"
format_stamp: "Formatted at 2026-06-19 20:31:45 on dist-test-slave-4m58"
server_key: "31877a2f3df911160acbbfed7c9aef7a"
server_key_iv: "6d6ea74d2fa6961894ba6200e8ab850f"
server_key_version: "encryptionkey@0"
I20260619 20:31:45.270679 5061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance:
uuid: "b6232f5e56bc4b51b381d466fa0d5137"
format_stamp: "Formatted at 2026-06-19 20:31:45 on dist-test-slave-4m58"
server_key: "31877a2f3df911160acbbfed7c9aef7a"
server_key_iv: "6d6ea74d2fa6961894ba6200e8ab850f"
server_key_version: "encryptionkey@0"
I20260619 20:31:45.272053 5061 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:31:45.273059 5076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:45.273314 5061 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:31:45.273427 5061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "b6232f5e56bc4b51b381d466fa0d5137"
format_stamp: "Formatted at 2026-06-19 20:31:45 on dist-test-slave-4m58"
server_key: "31877a2f3df911160acbbfed7c9aef7a"
server_key_iv: "6d6ea74d2fa6961894ba6200e8ab850f"
server_key_version: "encryptionkey@0"
I20260619 20:31:45.273500 5061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:31:45.294380 5061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:31:45.298110 5061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:31:45.298285 5061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:31:45.298547 5061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:31:45.298897 5061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:31:45.298939 5061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:45.298970 5061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:31:45.298986 5061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:45.305989 5061 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:43439
I20260619 20:31:45.306048 5189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:43439 every 8 connection(s)
I20260619 20:31:45.306404 5061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
I20260619 20:31:45.311549 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 5061
I20260619 20:31:45.311678 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance
I20260619 20:31:45.311901 4429 external_mini_cluster.cc:1428] Setting key 1bad500517d33b3c20e195c756b0c550
Jun 19 20:31:45 dist-test-slave-4m58 krb5kdc[4954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901105, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:31:45.319993 5039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:45.307419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:41779 (local address 127.4.83.126:37747)
0619 20:31:45.307487 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:45.307489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:45.307815 (+ 326us) server_negotiation.cc:407] Connection header received
0619 20:31:45.308873 (+ 1058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:45.308875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:45.308900 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:45.308994 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:45.310282 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.310889 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.312655 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.312959 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.313038 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:45.316126 (+ 3088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:45.316129 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:45.316129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:45.316147 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:45.317993 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:45.318673 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:45.318674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:45.318675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:45.318770 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:45.319105 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:45.319106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:45.319107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:45.319292 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:45.319374 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:45.319733 (+ 359us) server_negotiation.cc:299] Negotiation successful
0619 20:31:45.319875 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 20:31:45.320133 5192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:45.307318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:37747 (local address 127.4.83.65:41779)
0619 20:31:45.307589 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:45.307638 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:45.308684 (+ 1046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:45.309202 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:45.309210 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:45.309787 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:31:45.310057 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:45.310075 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.311050 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.311099 (+ 49us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:45.312239 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:45.312245 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.313087 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.313089 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:45.313286 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:45.314045 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:31:45.314055 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:31:45.315981 (+ 1926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:31:45.318197 (+ 2216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:45.318208 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:45.318211 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:45.318551 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:45.318879 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:45.318880 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:45.318880 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:45.319014 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:45.319387 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:31:45.319394 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:31:45.319549 (+ 155us) client_negotiation.cc:769] Sending connection context
0619 20:31:45.319647 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 20:31:45.319913 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":107,"threads_started":1}
I20260619 20:31:45.320933 5190 heartbeater.cc:347] Connected to a master server at 127.4.83.126:37747
I20260619 20:31:45.321070 5190 heartbeater.cc:464] Registering TS with master...
I20260619 20:31:45.321298 5190 heartbeater.cc:511] Master 127.4.83.126:37747 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.4.83.66@KRBTEST.COM; defaulting to no policy
I20260619 20:31:45.322098 4999 ts_manager.cc:194] Registered new tserver with Master: b6232f5e56bc4b51b381d466fa0d5137 (127.4.83.65:43439)
I20260619 20:31:45.322849 4999 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.65@KRBTEST.COM'} at 127.4.83.65:41779
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
I20260619 20:31:45.344285 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:0
--local_ip_for_outbound_sockets=127.4.83.66
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:37747
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:36883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:31:45.443270 5195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:31:45.443539 5195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:31:45.443568 5195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:31:45.444888 5195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:31:45.444948 5195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:31:45.445003 5195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:31:45.446683 5195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:36883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:37747
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:31:45.446954 5195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:31:45.447196 5195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:31:45.453276 5203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:45.453256 5200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:45.453264 5201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:45.453655 5195 server_base.cc:1034] running on GCE node
I20260619 20:31:45.454057 5195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:31:45.454430 5195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:31:45.455670 5195 hybrid_clock.cc:648] HybridClock initialized: now 1781901105455649 us; error 42 us; skew 500 ppm
Jun 19 20:31:45 dist-test-slave-4m58 krb5kdc[4954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901105, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:31:45.459115 5195 init.cc:377] Logged in from keytab as kudu/127.4.83.66@KRBTEST.COM (short username kudu)
I20260619 20:31:45.459743 5195 webserver.cc:460] Webserver started at http://127.4.83.66:39493/ using document root <none> and password file <none>
I20260619 20:31:45.460032 5195 fs_manager.cc:359] Metadata directory not provided
I20260619 20:31:45.460089 5195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:31:45.460242 5195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:31:45.461539 5195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-1/data/instance:
uuid: "0721a04966c949fdbdfc93288321aff0"
format_stamp: "Formatted at 2026-06-19 20:31:45 on dist-test-slave-4m58"
server_key: "d1039cbc324ee38bd188c6b9b2231890"
server_key_iv: "030fb5f9544528488e7ceb68aab06e89"
server_key_version: "encryptionkey@0"
I20260619 20:31:45.461939 5195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance:
uuid: "0721a04966c949fdbdfc93288321aff0"
format_stamp: "Formatted at 2026-06-19 20:31:45 on dist-test-slave-4m58"
server_key: "d1039cbc324ee38bd188c6b9b2231890"
server_key_iv: "030fb5f9544528488e7ceb68aab06e89"
server_key_version: "encryptionkey@0"
I20260619 20:31:45.463675 5195 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260619 20:31:45.465065 5210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:45.465456 5195 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:31:45.465591 5195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "0721a04966c949fdbdfc93288321aff0"
format_stamp: "Formatted at 2026-06-19 20:31:45 on dist-test-slave-4m58"
server_key: "d1039cbc324ee38bd188c6b9b2231890"
server_key_iv: "030fb5f9544528488e7ceb68aab06e89"
server_key_version: "encryptionkey@0"
I20260619 20:31:45.465700 5195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:31:45.475013 5195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:31:45.478179 5195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:31:45.478543 5195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:31:45.478938 5195 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:31:45.479301 5195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:31:45.479337 5195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:45.479368 5195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:31:45.479395 5195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:45.487118 5195 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:40695
I20260619 20:31:45.487214 5323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:40695 every 8 connection(s)
I20260619 20:31:45.487624 5195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
I20260619 20:31:45.491436 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 5195
I20260619 20:31:45.491539 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance
I20260619 20:31:45.491693 4429 external_mini_cluster.cc:1428] Setting key fb29b6961864c9a1fba2ec93980932ba
Jun 19 20:31:45 dist-test-slave-4m58 krb5kdc[4954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901105, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:31:45.500003 5039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:45.488632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:47313 (local address 127.4.83.126:37747)
0619 20:31:45.488743 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:45.488746 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:45.488935 (+ 189us) server_negotiation.cc:407] Connection header received
0619 20:31:45.490027 (+ 1092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:45.490029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:45.490052 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:45.490143 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:45.490959 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.491523 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.492954 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.493158 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.493226 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:45.496408 (+ 3182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:45.496411 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:45.496412 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:45.496423 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:45.498267 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:45.498849 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:45.498851 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:45.498851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:45.498920 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:45.499171 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:45.499171 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:45.499172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:45.499322 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:45.499386 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:45.499727 (+ 341us) server_negotiation.cc:299] Negotiation successful
0619 20:31:45.499886 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
WARNING: no policy specified for kudu/127.4.83.67@KRBTEST.COM; defaulting to no policy
I20260619 20:31:45.500238 5326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:45.488561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:37747 (local address 127.4.83.66:47313)
0619 20:31:45.488761 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:45.488806 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:45.489879 (+ 1073us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:45.490228 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:45.490233 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:45.490547 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:31:45.490778 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:45.490782 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.491656 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.491657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:45.492848 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:45.492851 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.493243 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.493243 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:45.493392 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:45.494102 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:31:45.494114 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:31:45.496153 (+ 2039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:31:45.498405 (+ 2252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:45.498424 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:45.498428 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:45.498765 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:45.498979 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:45.498980 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:45.498980 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:45.499112 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:45.499385 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:31:45.499394 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:31:45.499579 (+ 185us) client_negotiation.cc:769] Sending connection context
0619 20:31:45.499688 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 20:31:45.499950 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":94,"threads_started":1}
I20260619 20:31:45.500928 5324 heartbeater.cc:347] Connected to a master server at 127.4.83.126:37747
I20260619 20:31:45.501086 5324 heartbeater.cc:464] Registering TS with master...
I20260619 20:31:45.501298 5324 heartbeater.cc:511] Master 127.4.83.126:37747 requested a full tablet report, sending...
I20260619 20:31:45.501772 4999 ts_manager.cc:194] Registered new tserver with Master: 0721a04966c949fdbdfc93288321aff0 (127.4.83.66:40695)
I20260619 20:31:45.502236 4999 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.66@KRBTEST.COM'} at 127.4.83.66:47313
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
I20260619 20:31:45.519390 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:0
--local_ip_for_outbound_sockets=127.4.83.67
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:37747
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:36883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:31:45.613180 5329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:31:45.613454 5329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:31:45.613484 5329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:31:45.614943 5329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:31:45.615001 5329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:31:45.615032 5329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:31:45.617056 5329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:36883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:37747
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:31:45.617326 5329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:31:45.617542 5329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:31:45.622880 5334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:45.622943 5337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:45.623186 5329 server_base.cc:1034] running on GCE node
W20260619 20:31:45.622933 5335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:45.623632 5329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:31:45.623967 5329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:31:45.625152 5329 hybrid_clock.cc:648] HybridClock initialized: now 1781901105625124 us; error 47 us; skew 500 ppm
Jun 19 20:31:45 dist-test-slave-4m58 krb5kdc[4954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901105, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:31:45.628162 5329 init.cc:377] Logged in from keytab as kudu/127.4.83.67@KRBTEST.COM (short username kudu)
I20260619 20:31:45.628772 5329 webserver.cc:460] Webserver started at http://127.4.83.67:33163/ using document root <none> and password file <none>
I20260619 20:31:45.629036 5329 fs_manager.cc:359] Metadata directory not provided
I20260619 20:31:45.629462 5329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:31:45.629621 5329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:31:45.630720 5329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-2/data/instance:
uuid: "9c13dc4f84154c5c81aade3aaa823916"
format_stamp: "Formatted at 2026-06-19 20:31:45 on dist-test-slave-4m58"
server_key: "e13c982d5b2974b0dee290944080fc0d"
server_key_iv: "0c0b360c9ac43e6ff19e6a3f421710c0"
server_key_version: "encryptionkey@0"
I20260619 20:31:45.631031 5329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance:
uuid: "9c13dc4f84154c5c81aade3aaa823916"
format_stamp: "Formatted at 2026-06-19 20:31:45 on dist-test-slave-4m58"
server_key: "e13c982d5b2974b0dee290944080fc0d"
server_key_iv: "0c0b360c9ac43e6ff19e6a3f421710c0"
server_key_version: "encryptionkey@0"
I20260619 20:31:45.632541 5329 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:31:45.633517 5344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:45.633805 5329 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:31:45.633939 5329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "9c13dc4f84154c5c81aade3aaa823916"
format_stamp: "Formatted at 2026-06-19 20:31:45 on dist-test-slave-4m58"
server_key: "e13c982d5b2974b0dee290944080fc0d"
server_key_iv: "0c0b360c9ac43e6ff19e6a3f421710c0"
server_key_version: "encryptionkey@0"
I20260619 20:31:45.634001 5329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:31:45.647886 5329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:31:45.650158 5329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:31:45.650278 5329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:31:45.650498 5329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:31:45.650827 5329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:31:45.650874 5329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:45.650897 5329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:31:45.650908 5329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:45.657892 5329 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:39277
I20260619 20:31:45.658039 5457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:39277 every 8 connection(s)
I20260619 20:31:45.658325 5329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
Jun 19 20:31:45 dist-test-slave-4m58 krb5kdc[4954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901105, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:31:45.665405 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 5329
I20260619 20:31:45.665519 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance
I20260619 20:31:45.665668 4429 external_mini_cluster.cc:1428] Setting key cb16b20771035e9af4c8babe6aaad627
I20260619 20:31:45.669710 5039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:45.659207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:35611 (local address 127.4.83.126:37747)
0619 20:31:45.659269 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:45.659271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:45.659452 (+ 181us) server_negotiation.cc:407] Connection header received
0619 20:31:45.660473 (+ 1021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:45.660475 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:45.660487 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:45.660575 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:45.661220 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.661776 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.662849 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.663082 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.663155 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:45.665483 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:45.665484 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:45.665485 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:45.665493 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:45.667140 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:45.667653 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:45.667654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:45.667654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:45.667733 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:45.668850 (+ 1117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:45.668851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:45.668852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:45.669006 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:45.669083 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:45.669524 (+ 441us) server_negotiation.cc:299] Negotiation successful
0619 20:31:45.669636 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 20:31:45.669957 5460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:45.659174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:37747 (local address 127.4.83.67:35611)
0619 20:31:45.659332 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:45.659375 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:45.660353 (+ 978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:45.660631 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:45.660634 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:45.660898 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:31:45.661095 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:45.661098 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.661911 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.661912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:45.662741 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:45.662743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.663162 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.663163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:45.663266 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:45.663787 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:31:45.663796 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:31:45.665354 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:31:45.667233 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:45.667243 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:45.667245 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:45.667539 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:45.668572 (+ 1033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:45.668573 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:45.668574 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:45.668741 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:45.669119 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:31:45.669130 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:31:45.669319 (+ 189us) client_negotiation.cc:769] Sending connection context
0619 20:31:45.669470 (+ 151us) client_negotiation.cc:240] Negotiation successful
0619 20:31:45.669696 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":71,"threads_started":1}
I20260619 20:31:45.670600 5458 heartbeater.cc:347] Connected to a master server at 127.4.83.126:37747
I20260619 20:31:45.670727 5458 heartbeater.cc:464] Registering TS with master...
I20260619 20:31:45.670955 5458 heartbeater.cc:511] Master 127.4.83.126:37747 requested a full tablet report, sending...
I20260619 20:31:45.671430 4999 ts_manager.cc:194] Registered new tserver with Master: 9c13dc4f84154c5c81aade3aaa823916 (127.4.83.67:39277)
I20260619 20:31:45.671942 4999 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.67@KRBTEST.COM'} at 127.4.83.67:35611
I20260619 20:31:45.678220 4429 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:31:45.686861 5039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:45.680182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48132 (local address 127.4.83.126:37747)
0619 20:31:45.680237 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:45.680239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:45.680345 (+ 106us) server_negotiation.cc:407] Connection header received
0619 20:31:45.680411 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:45.680412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:45.680422 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:45.680487 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:45.681080 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.681591 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.682413 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.682617 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.682674 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:45.684044 (+ 1370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:45.684046 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:45.684046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:45.684053 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:45.685569 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:45.686016 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:45.686017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:45.686017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:45.686083 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:45.686300 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:45.686301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:45.686301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:45.686483 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:45.686582 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:45.686691 (+ 109us) server_negotiation.cc:299] Negotiation successful
0619 20:31:45.686789 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 20:31:45.689797 4999 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48132:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:31:45.691761 4999 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:31:45.699734 5472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:45.695421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:39277 (local address 127.0.0.1:50490)
0619 20:31:45.695713 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:45.695724 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:45.696033 (+ 309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:45.696338 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:45.696339 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:45.696341 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:31:45.696385 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:45.696386 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.697526 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.697528 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:45.698533 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:45.698536 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.699513 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.699514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:45.699575 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:45.699579 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:31:45.699622 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 20:31:45.699660 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":86,"threads_started":1}
I20260619 20:31:45.699818 5473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:45.695565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50490 (local address 127.4.83.67:39277)
0619 20:31:45.696160 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:45.696166 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:45.696176 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:31:45.696222 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:45.696225 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:45.696243 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:45.696330 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:31:45.696469 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.697415 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.698626 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.699421 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.699514 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:45.699592 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:45.699651 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 20:31:45.699709 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":98,"threads_started":1}
I20260619 20:31:45.699944 5474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:45.695727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52656 (local address 127.4.83.65:43439)
0619 20:31:45.695916 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:45.695920 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:45.695930 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:31:45.695965 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:45.695967 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:45.695981 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:45.696055 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:31:45.696250 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.697027 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.698764 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.699280 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.699344 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:45.699407 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:45.699775 (+ 368us) server_negotiation.cc:299] Negotiation successful
0619 20:31:45.699821 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":147,"threads_started":1}
I20260619 20:31:45.700217 5470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:45.695079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41174 (local address 127.4.83.66:40695)
0619 20:31:45.695238 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:45.695243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:45.695260 (+ 17us) server_negotiation.cc:407] Connection header received
0619 20:31:45.695415 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:45.695419 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:45.695436 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:45.695521 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:31:45.695822 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.697068 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.698767 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.699491 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.700022 (+ 531us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:45.700073 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:45.700095 (+ 22us) server_negotiation.cc:299] Negotiation successful
0619 20:31:45.700150 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
I20260619 20:31:45.699906 5469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:45.694971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:40695 (local address 127.0.0.1:41174)
0619 20:31:45.695153 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:45.695205 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:45.695291 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:45.695617 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:45.695621 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:45.695623 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:31:45.695710 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:45.695712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.697205 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.697206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:45.698656 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:45.698661 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.699597 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.699598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:45.699695 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:45.699702 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 20:31:45.699735 (+ 33us) client_negotiation.cc:240] Negotiation successful
0619 20:31:45.699795 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":68,"thread_start_us":77,"threads_started":1}
I20260619 20:31:45.699734 5471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:45.695211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:43439 (local address 127.0.0.1:52656)
0619 20:31:45.695391 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:45.695405 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:45.695471 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:45.696130 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:45.696131 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:45.696134 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:31:45.696181 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:45.696182 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.697123 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.697124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:45.698658 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:45.698661 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.699372 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.699373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:45.699474 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:45.699489 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 20:31:45.699562 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 20:31:45.699625 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":87,"thread_start_us":108,"threads_started":1}
I20260619 20:31:45.701126 5258 tablet_service.cc:1467] Processing CreateTablet for tablet cb5d86e52b874fad9fa8b03d8a645423 (DEFAULT_TABLE table=test-table [id=3a7a7305673b400f951e511bfa75f9de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:31:45.701141 5392 tablet_service.cc:1467] Processing CreateTablet for tablet cb5d86e52b874fad9fa8b03d8a645423 (DEFAULT_TABLE table=test-table [id=3a7a7305673b400f951e511bfa75f9de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:31:45.701505 5258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb5d86e52b874fad9fa8b03d8a645423. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:45.701525 5392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb5d86e52b874fad9fa8b03d8a645423. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:45.704481 5475 tablet_bootstrap.cc:492] T cb5d86e52b874fad9fa8b03d8a645423 P 0721a04966c949fdbdfc93288321aff0: Bootstrap starting.
I20260619 20:31:45.704489 5124 tablet_service.cc:1467] Processing CreateTablet for tablet cb5d86e52b874fad9fa8b03d8a645423 (DEFAULT_TABLE table=test-table [id=3a7a7305673b400f951e511bfa75f9de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:31:45.704882 5476 tablet_bootstrap.cc:492] T cb5d86e52b874fad9fa8b03d8a645423 P 9c13dc4f84154c5c81aade3aaa823916: Bootstrap starting.
I20260619 20:31:45.704938 5124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb5d86e52b874fad9fa8b03d8a645423. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:45.705339 5475 tablet_bootstrap.cc:654] T cb5d86e52b874fad9fa8b03d8a645423 P 0721a04966c949fdbdfc93288321aff0: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:45.705576 5476 tablet_bootstrap.cc:654] T cb5d86e52b874fad9fa8b03d8a645423 P 9c13dc4f84154c5c81aade3aaa823916: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:45.705952 5475 log.cc:824] T cb5d86e52b874fad9fa8b03d8a645423 P 0721a04966c949fdbdfc93288321aff0: Log is configured to *not* fsync() on all Append() calls
I20260619 20:31:45.706094 5476 log.cc:824] T cb5d86e52b874fad9fa8b03d8a645423 P 9c13dc4f84154c5c81aade3aaa823916: Log is configured to *not* fsync() on all Append() calls
I20260619 20:31:45.706898 5476 tablet_bootstrap.cc:492] T cb5d86e52b874fad9fa8b03d8a645423 P 9c13dc4f84154c5c81aade3aaa823916: No bootstrap required, opened a new log
I20260619 20:31:45.706965 5475 tablet_bootstrap.cc:492] T cb5d86e52b874fad9fa8b03d8a645423 P 0721a04966c949fdbdfc93288321aff0: No bootstrap required, opened a new log
I20260619 20:31:45.706982 5476 ts_tablet_manager.cc:1397] T cb5d86e52b874fad9fa8b03d8a645423 P 9c13dc4f84154c5c81aade3aaa823916: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:31:45.707033 5475 ts_tablet_manager.cc:1397] T cb5d86e52b874fad9fa8b03d8a645423 P 0721a04966c949fdbdfc93288321aff0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 20:31:45.709050 5475 raft_consensus.cc:348] T cb5d86e52b874fad9fa8b03d8a645423 P 0721a04966c949fdbdfc93288321aff0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0721a04966c949fdbdfc93288321aff0" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40695 } } peers { permanent_uuid: "b6232f5e56bc4b51b381d466fa0d5137" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 43439 } } peers { permanent_uuid: "9c13dc4f84154c5c81aade3aaa823916" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 39277 } }
I20260619 20:31:45.709050 5476 raft_consensus.cc:348] T cb5d86e52b874fad9fa8b03d8a645423 P 9c13dc4f84154c5c81aade3aaa823916 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0721a04966c949fdbdfc93288321aff0" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40695 } } peers { permanent_uuid: "b6232f5e56bc4b51b381d466fa0d5137" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 43439 } } peers { permanent_uuid: "9c13dc4f84154c5c81aade3aaa823916" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 39277 } }
I20260619 20:31:45.709264 5476 raft_consensus.cc:374] T cb5d86e52b874fad9fa8b03d8a645423 P 9c13dc4f84154c5c81aade3aaa823916 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:45.709264 5475 raft_consensus.cc:374] T cb5d86e52b874fad9fa8b03d8a645423 P 0721a04966c949fdbdfc93288321aff0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:45.709300 5475 raft_consensus.cc:729] T cb5d86e52b874fad9fa8b03d8a645423 P 0721a04966c949fdbdfc93288321aff0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0721a04966c949fdbdfc93288321aff0, State: Initialized, Role: FOLLOWER
I20260619 20:31:45.709300 5476 raft_consensus.cc:729] T cb5d86e52b874fad9fa8b03d8a645423 P 9c13dc4f84154c5c81aade3aaa823916 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c13dc4f84154c5c81aade3aaa823916, State: Initialized, Role: FOLLOWER
I20260619 20:31:45.709441 5475 consensus_queue.cc:260] T cb5d86e52b874fad9fa8b03d8a645423 P 0721a04966c949fdbdfc93288321aff0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0721a04966c949fdbdfc93288321aff0" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40695 } } peers { permanent_uuid: "b6232f5e56bc4b51b381d466fa0d5137" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 43439 } } peers { permanent_uuid: "9c13dc4f84154c5c81aade3aaa823916" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 39277 } }
I20260619 20:31:45.709441 5476 consensus_queue.cc:260] T cb5d86e52b874fad9fa8b03d8a645423 P 9c13dc4f84154c5c81aade3aaa823916 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0721a04966c949fdbdfc93288321aff0" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40695 } } peers { permanent_uuid: "b6232f5e56bc4b51b381d466fa0d5137" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 43439 } } peers { permanent_uuid: "9c13dc4f84154c5c81aade3aaa823916" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 39277 } }
I20260619 20:31:45.709760 5475 ts_tablet_manager.cc:1428] T cb5d86e52b874fad9fa8b03d8a645423 P 0721a04966c949fdbdfc93288321aff0: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 20:31:45.709760 5476 ts_tablet_manager.cc:1428] T cb5d86e52b874fad9fa8b03d8a645423 P 9c13dc4f84154c5c81aade3aaa823916: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 20:31:45.709839 5458 heartbeater.cc:503] Master 127.4.83.126:37747 was elected leader, sending a full tablet report...
I20260619 20:31:45.709911 5324 heartbeater.cc:503] Master 127.4.83.126:37747 was elected leader, sending a full tablet report...
I20260619 20:31:45.710035 5479 tablet_bootstrap.cc:492] T cb5d86e52b874fad9fa8b03d8a645423 P b6232f5e56bc4b51b381d466fa0d5137: Bootstrap starting.
I20260619 20:31:45.710803 5479 tablet_bootstrap.cc:654] T cb5d86e52b874fad9fa8b03d8a645423 P b6232f5e56bc4b51b381d466fa0d5137: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:45.711375 5479 log.cc:824] T cb5d86e52b874fad9fa8b03d8a645423 P b6232f5e56bc4b51b381d466fa0d5137: Log is configured to *not* fsync() on all Append() calls
I20260619 20:31:45.712304 5479 tablet_bootstrap.cc:492] T cb5d86e52b874fad9fa8b03d8a645423 P b6232f5e56bc4b51b381d466fa0d5137: No bootstrap required, opened a new log
I20260619 20:31:45.712378 5479 ts_tablet_manager.cc:1397] T cb5d86e52b874fad9fa8b03d8a645423 P b6232f5e56bc4b51b381d466fa0d5137: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:31:45.713850 5479 raft_consensus.cc:348] T cb5d86e52b874fad9fa8b03d8a645423 P b6232f5e56bc4b51b381d466fa0d5137 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0721a04966c949fdbdfc93288321aff0" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40695 } } peers { permanent_uuid: "b6232f5e56bc4b51b381d466fa0d5137" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 43439 } } peers { permanent_uuid: "9c13dc4f84154c5c81aade3aaa823916" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 39277 } }
I20260619 20:31:45.714010 5479 raft_consensus.cc:374] T cb5d86e52b874fad9fa8b03d8a645423 P b6232f5e56bc4b51b381d466fa0d5137 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:45.714031 5479 raft_consensus.cc:729] T cb5d86e52b874fad9fa8b03d8a645423 P b6232f5e56bc4b51b381d466fa0d5137 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6232f5e56bc4b51b381d466fa0d5137, State: Initialized, Role: FOLLOWER
I20260619 20:31:45.714140 5479 consensus_queue.cc:260] T cb5d86e52b874fad9fa8b03d8a645423 P b6232f5e56bc4b51b381d466fa0d5137 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0721a04966c949fdbdfc93288321aff0" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40695 } } peers { permanent_uuid: "b6232f5e56bc4b51b381d466fa0d5137" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 43439 } } peers { permanent_uuid: "9c13dc4f84154c5c81aade3aaa823916" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 39277 } }
I20260619 20:31:45.714447 5479 ts_tablet_manager.cc:1428] T cb5d86e52b874fad9fa8b03d8a645423 P b6232f5e56bc4b51b381d466fa0d5137: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:31:45.714509 5190 heartbeater.cc:503] Master 127.4.83.126:37747 was elected leader, sending a full tablet report...
W20260619 20:31:45.738400 5325 tablet.cc:2369] T cb5d86e52b874fad9fa8b03d8a645423 P 0721a04966c949fdbdfc93288321aff0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:31:45.772879 5481 raft_consensus.cc:482] T cb5d86e52b874fad9fa8b03d8a645423 P 0721a04966c949fdbdfc93288321aff0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:31:45.773110 5481 raft_consensus.cc:504] T cb5d86e52b874fad9fa8b03d8a645423 P 0721a04966c949fdbdfc93288321aff0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0721a04966c949fdbdfc93288321aff0" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40695 } } peers { permanent_uuid: "b6232f5e56bc4b51b381d466fa0d5137" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 43439 } } peers { permanent_uuid: "9c13dc4f84154c5c81aade3aaa823916" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 39277 } }
I20260619 20:31:45.773612 5481 leader_election.cc:288] T cb5d86e52b874fad9fa8b03d8a645423 P 0721a04966c949fdbdfc93288321aff0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6232f5e56bc4b51b381d466fa0d5137 (127.4.83.65:43439), 9c13dc4f84154c5c81aade3aaa823916 (127.4.83.67:39277)
I20260619 20:31:45.777534 5473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:45.774099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:47949 (local address 127.4.83.67:39277)
0619 20:31:45.774188 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:45.774191 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:45.774297 (+ 106us) server_negotiation.cc:407] Connection header received
0619 20:31:45.774390 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:45.774391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:45.774406 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:45.774453 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:31:45.774657 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.775277 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.776679 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.777113 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.777196 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:45.777220 (+ 24us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:45.777404 (+ 184us) server_negotiation.cc:299] Negotiation successful
0619 20:31:45.777453 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 20:31:45.777537 5474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:45.773910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:42547 (local address 127.4.83.65:43439)
0619 20:31:45.773969 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:45.773971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:45.773988 (+ 17us) server_negotiation.cc:407] Connection header received
0619 20:31:45.774066 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:45.774068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:45.774083 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:45.774185 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:31:45.774394 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.775095 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.776638 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.777091 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.777208 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:45.777220 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:45.777391 (+ 171us) server_negotiation.cc:299] Negotiation successful
0619 20:31:45.777436 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 20:31:45.777603 5326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:45.773815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:43439 (local address 127.4.83.66:42547)
0619 20:31:45.773913 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:45.773928 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:45.773979 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:45.774212 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:45.774213 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:45.774216 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:31:45.774301 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:45.774304 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.775159 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.775160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:45.776518 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:45.776521 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.777216 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.777218 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:45.777326 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:45.777330 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:31:45.777380 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 20:31:45.777463 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70,"mutex_wait_us":4}
I20260619 20:31:45.777630 5484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:45.774002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:39277 (local address 127.4.83.66:47949)
0619 20:31:45.774207 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:45.774220 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:45.774293 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:45.774473 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:45.774474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:45.774476 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:31:45.774580 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:45.774583 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.775356 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.775357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:45.776530 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:45.776532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.777271 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.777272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:45.777334 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:45.777338 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:31:45.777386 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 20:31:45.777443 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":131,"threads_started":1}
I20260619 20:31:45.777972 5412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb5d86e52b874fad9fa8b03d8a645423" candidate_uuid: "0721a04966c949fdbdfc93288321aff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c13dc4f84154c5c81aade3aaa823916" is_pre_election: true
I20260619 20:31:45.778156 5412 raft_consensus.cc:2454] T cb5d86e52b874fad9fa8b03d8a645423 P 9c13dc4f84154c5c81aade3aaa823916 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0721a04966c949fdbdfc93288321aff0 in term 0.
I20260619 20:31:45.778159 5144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb5d86e52b874fad9fa8b03d8a645423" candidate_uuid: "0721a04966c949fdbdfc93288321aff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6232f5e56bc4b51b381d466fa0d5137" is_pre_election: true
I20260619 20:31:45.778393 5144 raft_consensus.cc:2454] T cb5d86e52b874fad9fa8b03d8a645423 P b6232f5e56bc4b51b381d466fa0d5137 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0721a04966c949fdbdfc93288321aff0 in term 0.
I20260619 20:31:45.778446 5213 leader_election.cc:302] T cb5d86e52b874fad9fa8b03d8a645423 P 0721a04966c949fdbdfc93288321aff0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0721a04966c949fdbdfc93288321aff0, 9c13dc4f84154c5c81aade3aaa823916; no voters:
I20260619 20:31:45.778678 5481 raft_consensus.cc:2781] T cb5d86e52b874fad9fa8b03d8a645423 P 0721a04966c949fdbdfc93288321aff0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:31:45.778733 5481 raft_consensus.cc:482] T cb5d86e52b874fad9fa8b03d8a645423 P 0721a04966c949fdbdfc93288321aff0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:31:45.778755 5481 raft_consensus.cc:3037] T cb5d86e52b874fad9fa8b03d8a645423 P 0721a04966c949fdbdfc93288321aff0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:45.779507 5481 raft_consensus.cc:504] T cb5d86e52b874fad9fa8b03d8a645423 P 0721a04966c949fdbdfc93288321aff0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0721a04966c949fdbdfc93288321aff0" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40695 } } peers { permanent_uuid: "b6232f5e56bc4b51b381d466fa0d5137" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 43439 } } peers { permanent_uuid: "9c13dc4f84154c5c81aade3aaa823916" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 39277 } }
I20260619 20:31:45.779642 5481 leader_election.cc:288] T cb5d86e52b874fad9fa8b03d8a645423 P 0721a04966c949fdbdfc93288321aff0 [CANDIDATE]: Term 1 election: Requested vote from peers b6232f5e56bc4b51b381d466fa0d5137 (127.4.83.65:43439), 9c13dc4f84154c5c81aade3aaa823916 (127.4.83.67:39277)
I20260619 20:31:45.779904 5144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb5d86e52b874fad9fa8b03d8a645423" candidate_uuid: "0721a04966c949fdbdfc93288321aff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6232f5e56bc4b51b381d466fa0d5137"
I20260619 20:31:45.779978 5144 raft_consensus.cc:3037] T cb5d86e52b874fad9fa8b03d8a645423 P b6232f5e56bc4b51b381d466fa0d5137 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:45.779944 5412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb5d86e52b874fad9fa8b03d8a645423" candidate_uuid: "0721a04966c949fdbdfc93288321aff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c13dc4f84154c5c81aade3aaa823916"
I20260619 20:31:45.780143 5412 raft_consensus.cc:3037] T cb5d86e52b874fad9fa8b03d8a645423 P 9c13dc4f84154c5c81aade3aaa823916 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:45.780790 5144 raft_consensus.cc:2454] T cb5d86e52b874fad9fa8b03d8a645423 P b6232f5e56bc4b51b381d466fa0d5137 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0721a04966c949fdbdfc93288321aff0 in term 1.
I20260619 20:31:45.780958 5213 leader_election.cc:302] T cb5d86e52b874fad9fa8b03d8a645423 P 0721a04966c949fdbdfc93288321aff0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0721a04966c949fdbdfc93288321aff0, b6232f5e56bc4b51b381d466fa0d5137; no voters:
I20260619 20:31:45.781059 5481 raft_consensus.cc:2781] T cb5d86e52b874fad9fa8b03d8a645423 P 0721a04966c949fdbdfc93288321aff0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:31:45.781210 5481 raft_consensus.cc:686] T cb5d86e52b874fad9fa8b03d8a645423 P 0721a04966c949fdbdfc93288321aff0 [term 1 LEADER]: Becoming Leader. State: Replica: 0721a04966c949fdbdfc93288321aff0, State: Running, Role: LEADER
I20260619 20:31:45.781330 5481 consensus_queue.cc:237] T cb5d86e52b874fad9fa8b03d8a645423 P 0721a04966c949fdbdfc93288321aff0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0721a04966c949fdbdfc93288321aff0" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40695 } } peers { permanent_uuid: "b6232f5e56bc4b51b381d466fa0d5137" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 43439 } } peers { permanent_uuid: "9c13dc4f84154c5c81aade3aaa823916" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 39277 } }
I20260619 20:31:45.781415 5412 raft_consensus.cc:2454] T cb5d86e52b874fad9fa8b03d8a645423 P 9c13dc4f84154c5c81aade3aaa823916 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0721a04966c949fdbdfc93288321aff0 in term 1.
I20260619 20:31:45.782351 4998 catalog_manager.cc:5495] T cb5d86e52b874fad9fa8b03d8a645423 P 0721a04966c949fdbdfc93288321aff0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0721a04966c949fdbdfc93288321aff0 (127.4.83.66). New cstate: current_term: 1 leader_uuid: "0721a04966c949fdbdfc93288321aff0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0721a04966c949fdbdfc93288321aff0" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6232f5e56bc4b51b381d466fa0d5137" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 43439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c13dc4f84154c5c81aade3aaa823916" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 39277 } health_report { overall_health: UNKNOWN } } }
Jun 19 20:31:45 dist-test-slave-4m58 krb5kdc[4954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.65@KRBTEST.COM
I20260619 20:31:45.805637 5474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:45.795269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52672 (local address 127.4.83.65:43439)
0619 20:31:45.795321 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:45.795323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:45.795595 (+ 272us) server_negotiation.cc:407] Connection header received
0619 20:31:45.795644 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:45.795645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:45.795660 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:45.795730 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:45.796412 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.796991 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.798012 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.798285 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.798379 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:45.801528 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:45.801550 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:45.801550 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:45.801562 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:45.804091 (+ 2529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:45.804649 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:45.804655 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:45.804655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:45.804746 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:45.805040 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:45.805041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:45.805041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:45.805245 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:45.805382 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:45.805439 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 20:31:45.805537 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260619 20:31:45.807374 5191 tablet.cc:2369] T cb5d86e52b874fad9fa8b03d8a645423 P b6232f5e56bc4b51b381d466fa0d5137: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 19 20:31:45 dist-test-slave-4m58 krb5kdc[4954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901105, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 20:31:45 dist-test-slave-4m58 krb5kdc[4954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901105, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.83.65@KRBTEST.COM
I20260619 20:31:45.834364 5474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:45.825462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52678 (local address 127.4.83.65:43439)
0619 20:31:45.825531 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:45.825533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:45.825609 (+ 76us) server_negotiation.cc:407] Connection header received
0619 20:31:45.825699 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:45.825700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:45.825712 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:45.825763 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:45.826504 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.827069 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.827951 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.828166 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.828228 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:45.830859 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:45.830861 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:45.830861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:45.830871 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:45.832737 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:45.833227 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:45.833228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:45.833229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:45.833322 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:45.833620 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:45.833622 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:45.833622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:45.833790 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:45.833912 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:45.834150 (+ 238us) server_negotiation.cc:299] Negotiation successful
0619 20:31:45.834273 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
Jun 19 20:31:45 dist-test-slave-4m58 krb5kdc[4954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 20:31:45 dist-test-slave-4m58 krb5kdc[4954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.65@KRBTEST.COM
I20260619 20:31:45.861824 5474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:45.853218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52690 (local address 127.4.83.65:43439)
0619 20:31:45.853357 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:45.853360 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:45.853443 (+ 83us) server_negotiation.cc:407] Connection header received
0619 20:31:45.853523 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:45.853524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:45.853535 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:45.853584 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:45.854202 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.854714 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.855497 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:45.855695 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:45.855754 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:45.858361 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:45.858364 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:45.858365 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:45.858377 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:45.860337 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:45.860894 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:45.860896 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:45.860896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:45.861004 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:45.861256 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:45.861256 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:45.861257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:45.861409 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:45.861519 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:45.861645 (+ 126us) server_negotiation.cc:299] Negotiation successful
0619 20:31:45.861732 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260619 20:31:45.862361 5104 tablet_service.cc:2908] Rejecting scan request for tablet cb5d86e52b874fad9fa8b03d8a645423: Uninitialized: clean time has not yet been initialized
I20260619 20:31:45.863430 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 5061
I20260619 20:31:45.869200 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 5195
I20260619 20:31:45.874212 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 5329
I20260619 20:31:45.878809 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 4967
2026-06-19T20:31:45Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (1998 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:31:45 dist-test-slave-4m58 krb5kdc[5499](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:31:45 dist-test-slave-4m58 krb5kdc[5499](info): set up 2 sockets
Jun 19 20:31:45 dist-test-slave-4m58 krb5kdc[5499](info): commencing operation
krb5kdc: starting...
W20260619 20:31:46.880905 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.981s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:31:46 dist-test-slave-4m58 krb5kdc[5499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:31:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:31:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
I20260619 20:31:47.033118 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:38939
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:41271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:38939
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:31:47.157601 5512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:31:47.157878 5512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:31:47.157903 5512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:31:47.159278 5512 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:31:47.159474 5512 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 20:31:47.159489 5512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:31:47.159507 5512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:31:47.159519 5512 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:31:47.159533 5512 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:31:47.161520 5512 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:41271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:38939
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:38939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:31:47.161834 5512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:31:47.162107 5512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:31:47.168727 5512 server_base.cc:1034] running on GCE node
W20260619 20:31:47.168752 5517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:47.168777 5520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:47.168777 5518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:47.169493 5512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:31:47.169859 5512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:31:47.171096 5512 hybrid_clock.cc:648] HybridClock initialized: now 1781901107171059 us; error 56 us; skew 500 ppm
Jun 19 20:31:47 dist-test-slave-4m58 krb5kdc[5499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901107, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:31:47.178747 5512 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:31:47.179450 5512 webserver.cc:460] Webserver started at http://127.4.83.126:36617/ using document root <none> and password file <none>
I20260619 20:31:47.179715 5512 fs_manager.cc:359] Metadata directory not provided
I20260619 20:31:47.179772 5512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:31:47.179957 5512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:31:47.181154 5512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "11f488cd7e054da6ae03786d0972c1f6"
format_stamp: "Formatted at 2026-06-19 20:31:47 on dist-test-slave-4m58"
server_key: "fda29ef7e412087d216eabb32407990a"
server_key_iv: "7def5578809322de479ced1fd6e6b5f3"
server_key_version: "encryptionkey@0"
I20260619 20:31:47.181680 5512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "11f488cd7e054da6ae03786d0972c1f6"
format_stamp: "Formatted at 2026-06-19 20:31:47 on dist-test-slave-4m58"
server_key: "fda29ef7e412087d216eabb32407990a"
server_key_iv: "7def5578809322de479ced1fd6e6b5f3"
server_key_version: "encryptionkey@0"
I20260619 20:31:47.184357 5512 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260619 20:31:47.186532 5527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:47.186939 5512 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.001s
I20260619 20:31:47.187104 5512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "11f488cd7e054da6ae03786d0972c1f6"
format_stamp: "Formatted at 2026-06-19 20:31:47 on dist-test-slave-4m58"
server_key: "fda29ef7e412087d216eabb32407990a"
server_key_iv: "7def5578809322de479ced1fd6e6b5f3"
server_key_version: "encryptionkey@0"
I20260619 20:31:47.187201 5512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:31:47.198400 5512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:31:47.201109 5512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:31:47.201261 5512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:31:47.206589 5512 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:38939
I20260619 20:31:47.206759 5589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:38939 every 8 connection(s)
I20260619 20:31:47.207044 5512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:31:47.207965 5590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:47.210846 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 5512
I20260619 20:31:47.210985 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:31:47.211158 4429 external_mini_cluster.cc:1428] Setting key d788b4ddce3822570b4481990e2db320
I20260619 20:31:47.211519 5590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11f488cd7e054da6ae03786d0972c1f6: Bootstrap starting.
I20260619 20:31:47.212574 5590 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11f488cd7e054da6ae03786d0972c1f6: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:47.212965 5590 log.cc:824] T 00000000000000000000000000000000 P 11f488cd7e054da6ae03786d0972c1f6: Log is configured to *not* fsync() on all Append() calls
I20260619 20:31:47.213955 5590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11f488cd7e054da6ae03786d0972c1f6: No bootstrap required, opened a new log
I20260619 20:31:47.216285 5590 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11f488cd7e054da6ae03786d0972c1f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11f488cd7e054da6ae03786d0972c1f6" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 38939 } }
I20260619 20:31:47.216511 5590 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11f488cd7e054da6ae03786d0972c1f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:47.216548 5590 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11f488cd7e054da6ae03786d0972c1f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f488cd7e054da6ae03786d0972c1f6, State: Initialized, Role: FOLLOWER
I20260619 20:31:47.216722 5590 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11f488cd7e054da6ae03786d0972c1f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11f488cd7e054da6ae03786d0972c1f6" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 38939 } }
I20260619 20:31:47.216821 5590 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11f488cd7e054da6ae03786d0972c1f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:31:47.216850 5590 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11f488cd7e054da6ae03786d0972c1f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:31:47.216909 5590 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11f488cd7e054da6ae03786d0972c1f6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:47.217898 5590 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11f488cd7e054da6ae03786d0972c1f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11f488cd7e054da6ae03786d0972c1f6" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 38939 } }
I20260619 20:31:47.218103 5590 leader_election.cc:302] T 00000000000000000000000000000000 P 11f488cd7e054da6ae03786d0972c1f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11f488cd7e054da6ae03786d0972c1f6; no voters:
Jun 19 20:31:47 dist-test-slave-4m58 krb5kdc[5499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:31:47.218372 5590 leader_election.cc:288] T 00000000000000000000000000000000 P 11f488cd7e054da6ae03786d0972c1f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:31:47.218437 5595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11f488cd7e054da6ae03786d0972c1f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:31:47.218776 5595 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11f488cd7e054da6ae03786d0972c1f6 [term 1 LEADER]: Becoming Leader. State: Replica: 11f488cd7e054da6ae03786d0972c1f6, State: Running, Role: LEADER
I20260619 20:31:47.218906 5595 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11f488cd7e054da6ae03786d0972c1f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11f488cd7e054da6ae03786d0972c1f6" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 38939 } }
I20260619 20:31:47.218958 5590 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11f488cd7e054da6ae03786d0972c1f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:31:47.219370 5596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11f488cd7e054da6ae03786d0972c1f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11f488cd7e054da6ae03786d0972c1f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11f488cd7e054da6ae03786d0972c1f6" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 38939 } } }
I20260619 20:31:47.219455 5596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11f488cd7e054da6ae03786d0972c1f6 [sys.catalog]: This master's current role is: LEADER
I20260619 20:31:47.219624 5597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11f488cd7e054da6ae03786d0972c1f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11f488cd7e054da6ae03786d0972c1f6. Latest consensus state: current_term: 1 leader_uuid: "11f488cd7e054da6ae03786d0972c1f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11f488cd7e054da6ae03786d0972c1f6" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 38939 } } }
I20260619 20:31:47.219684 5597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11f488cd7e054da6ae03786d0972c1f6 [sys.catalog]: This master's current role is: LEADER
I20260619 20:31:47.220038 5599 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:31:47.220746 5599 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:31:47.223222 5599 catalog_manager.cc:1347] Generated new cluster ID: 89bef93616ee4962ad7a04cc5f607b1c
I20260619 20:31:47.223300 5599 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:31:47.224625 5593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:47.211792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56656 (local address 127.4.83.126:38939)
0619 20:31:47.212090 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:47.212102 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:47.212139 (+ 37us) server_negotiation.cc:407] Connection header received
0619 20:31:47.212808 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:47.212818 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:47.212879 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:47.213025 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:47.213780 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.214773 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.215508 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.216024 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.216141 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:47.219029 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:47.219034 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:47.219035 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:47.219050 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:47.221976 (+ 2926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:47.222688 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:47.222691 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:47.222691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:47.222898 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:47.223230 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:47.223231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:47.223232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:47.223723 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:47.223961 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:47.224221 (+ 260us) server_negotiation.cc:299] Negotiation successful
0619 20:31:47.224391 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":154,"threads_started":1}
I20260619 20:31:47.234927 5599 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:31:47.235603 5599 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:31:47.240689 5599 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11f488cd7e054da6ae03786d0972c1f6: Generated new TSK 0
I20260619 20:31:47.240947 5599 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.83.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
I20260619 20:31:47.272830 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:0
--local_ip_for_outbound_sockets=127.4.83.65
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:38939
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:41271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:31:47.368705 5616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:31:47.369144 5616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:31:47.369225 5616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:31:47.370700 5616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:31:47.370769 5616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:31:47.370802 5616 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 20:31:47.370827 5616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:31:47.372718 5616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:41271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.83.126:38939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:31:47.373018 5616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:31:47.373381 5616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:31:47.379024 5624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:47.379142 5622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:47.379041 5621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:47.379606 5616 server_base.cc:1034] running on GCE node
I20260619 20:31:47.379861 5616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:31:47.380132 5616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:31:47.381328 5616 hybrid_clock.cc:648] HybridClock initialized: now 1781901107381304 us; error 38 us; skew 500 ppm
Jun 19 20:31:47 dist-test-slave-4m58 krb5kdc[5499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901107, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:31:47.384898 5616 init.cc:377] Logged in from keytab as kudu/127.4.83.65@KRBTEST.COM (short username kudu)
I20260619 20:31:47.385449 5616 webserver.cc:460] Webserver started at http://127.4.83.65:32793/ using document root <none> and password file <none>
I20260619 20:31:47.385715 5616 fs_manager.cc:359] Metadata directory not provided
I20260619 20:31:47.385776 5616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:31:47.385911 5616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:31:47.387051 5616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-0/data/instance:
uuid: "9a028fe2ca8b4236a5d5f66dca0db482"
format_stamp: "Formatted at 2026-06-19 20:31:47 on dist-test-slave-4m58"
server_key: "3fb0d3306d669538c7412c02fbfab414"
server_key_iv: "1502daae6a31fd2937451ef502fb4430"
server_key_version: "encryptionkey@0"
I20260619 20:31:47.387390 5616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance:
uuid: "9a028fe2ca8b4236a5d5f66dca0db482"
format_stamp: "Formatted at 2026-06-19 20:31:47 on dist-test-slave-4m58"
server_key: "3fb0d3306d669538c7412c02fbfab414"
server_key_iv: "1502daae6a31fd2937451ef502fb4430"
server_key_version: "encryptionkey@0"
I20260619 20:31:47.389038 5616 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:31:47.390316 5631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:47.390605 5616 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:31:47.390710 5616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "9a028fe2ca8b4236a5d5f66dca0db482"
format_stamp: "Formatted at 2026-06-19 20:31:47 on dist-test-slave-4m58"
server_key: "3fb0d3306d669538c7412c02fbfab414"
server_key_iv: "1502daae6a31fd2937451ef502fb4430"
server_key_version: "encryptionkey@0"
I20260619 20:31:47.390810 5616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:31:47.412003 5616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:31:47.414481 5616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:31:47.414618 5616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:31:47.415212 5616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:31:47.415253 5616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:47.415292 5616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:31:47.415315 5616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 20:31:47 dist-test-slave-4m58 krb5kdc[5499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901107, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:31:47.424960 5616 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:34491
I20260619 20:31:47.425120 5746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:34491 every 8 connection(s)
I20260619 20:31:47.425518 5616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
I20260619 20:31:47.429556 5593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:47.415961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:42953 (local address 127.4.83.126:38939)
0619 20:31:47.416017 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:47.416019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:47.416392 (+ 373us) server_negotiation.cc:407] Connection header received
0619 20:31:47.417072 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:47.417075 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:47.417092 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:47.417184 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:47.418180 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.419020 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.420742 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.421088 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.421179 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:47.425309 (+ 4130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:47.425311 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:47.425311 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:47.425321 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:47.427140 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:47.427908 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:47.427910 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:47.427911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:47.427998 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:47.428609 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:47.428610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:47.428611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:47.428776 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:47.428842 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:47.429351 (+ 509us) server_negotiation.cc:299] Negotiation successful
0619 20:31:47.429469 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 20:31:47.429854 5640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:47.415952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:38939 (local address 127.4.83.65:42953)
0619 20:31:47.416221 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:47.416266 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:47.416899 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:47.417279 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:47.417284 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:47.417716 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:31:47.418047 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:47.418053 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.419501 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.419502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:47.420631 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:47.420634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.421246 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.421248 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:47.421499 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:47.422292 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:31:47.422307 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:31:47.425130 (+ 2823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:31:47.427297 (+ 2167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:47.427305 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:47.427307 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:47.427713 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:47.428150 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:47.428151 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:47.428152 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:47.428291 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:47.428850 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:31:47.428855 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:31:47.429020 (+ 165us) client_negotiation.cc:769] Sending connection context
0619 20:31:47.429267 (+ 247us) client_negotiation.cc:240] Negotiation successful
0619 20:31:47.429483 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":123,"threads_started":1}
I20260619 20:31:47.430332 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 5616
I20260619 20:31:47.430469 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance
I20260619 20:31:47.430641 4429 external_mini_cluster.cc:1428] Setting key 159af91a474cbf12ed6b0628d1d09e3e
I20260619 20:31:47.430840 5747 heartbeater.cc:347] Connected to a master server at 127.4.83.126:38939
I20260619 20:31:47.430959 5747 heartbeater.cc:464] Registering TS with master...
I20260619 20:31:47.431229 5747 heartbeater.cc:511] Master 127.4.83.126:38939 requested a full tablet report, sending...
I20260619 20:31:47.431963 5543 ts_manager.cc:194] Registered new tserver with Master: 9a028fe2ca8b4236a5d5f66dca0db482 (127.4.83.65:34491)
I20260619 20:31:47.432761 5543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.65@KRBTEST.COM'} at 127.4.83.65:42953
I20260619 20:31:47.440696 5593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:47.432106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:50869 (local address 127.4.83.126:38939)
0619 20:31:47.432165 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:47.432167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:47.433118 (+ 951us) server_negotiation.cc:407] Connection header received
0619 20:31:47.433187 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:47.433188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:47.433201 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:47.433251 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:47.433800 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.434538 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.435350 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.435631 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.435717 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:47.436999 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:47.437002 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:47.437003 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:47.437014 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:47.439249 (+ 2235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:47.439821 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:47.439823 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:47.439823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:47.439939 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:47.440131 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:47.440132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:47.440133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:47.440268 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:47.440332 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:47.440505 (+ 173us) server_negotiation.cc:299] Negotiation successful
0619 20:31:47.440601 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 20:31:47.440723 5753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:47.432207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:38939 (local address 127.4.83.65:50869)
0619 20:31:47.433035 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:47.433048 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:47.433125 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:47.433309 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:47.433311 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:47.433593 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:31:47.433710 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:47.433713 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.434668 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.434670 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:47.435247 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:47.435250 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.435741 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.435742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:47.435819 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:47.436531 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:31:47.436535 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:31:47.436871 (+ 336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:31:47.439434 (+ 2563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:47.439436 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:47.439436 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:47.439689 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:47.440010 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:47.440010 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:47.440011 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:47.440075 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:47.440328 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:31:47.440329 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:31:47.440384 (+ 55us) client_negotiation.cc:769] Sending connection context
0619 20:31:47.440444 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 20:31:47.440592 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":790,"thread_start_us":122,"threads_started":1}
WARNING: no policy specified for kudu/127.4.83.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
I20260619 20:31:47.464529 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:0
--local_ip_for_outbound_sockets=127.4.83.66
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:38939
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:41271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:31:47.567183 5756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:31:47.567485 5756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:31:47.567517 5756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:31:47.568953 5756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:31:47.569026 5756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:31:47.569061 5756 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 20:31:47.569080 5756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:31:47.570866 5756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:41271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.83.126:38939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:31:47.571187 5756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:31:47.571456 5756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:31:47.578802 5761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:47.578824 5762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:47.579078 5756 server_base.cc:1034] running on GCE node
W20260619 20:31:47.578768 5764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:47.579561 5756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:31:47.580013 5756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:31:47.581302 5756 hybrid_clock.cc:648] HybridClock initialized: now 1781901107581244 us; error 81 us; skew 500 ppm
Jun 19 20:31:47 dist-test-slave-4m58 krb5kdc[5499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901107, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:31:47.585572 5756 init.cc:377] Logged in from keytab as kudu/127.4.83.66@KRBTEST.COM (short username kudu)
I20260619 20:31:47.586248 5756 webserver.cc:460] Webserver started at http://127.4.83.66:39139/ using document root <none> and password file <none>
I20260619 20:31:47.586505 5756 fs_manager.cc:359] Metadata directory not provided
I20260619 20:31:47.586563 5756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:31:47.586709 5756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:31:47.588088 5756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-1/data/instance:
uuid: "bed13b3e786a4d2da073947f19d78d51"
format_stamp: "Formatted at 2026-06-19 20:31:47 on dist-test-slave-4m58"
server_key: "f7b7c4b806da18d440d01980b0a19c5d"
server_key_iv: "1e2773c9473b770daa9a6a66c0b9ea4e"
server_key_version: "encryptionkey@0"
I20260619 20:31:47.588478 5756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance:
uuid: "bed13b3e786a4d2da073947f19d78d51"
format_stamp: "Formatted at 2026-06-19 20:31:47 on dist-test-slave-4m58"
server_key: "f7b7c4b806da18d440d01980b0a19c5d"
server_key_iv: "1e2773c9473b770daa9a6a66c0b9ea4e"
server_key_version: "encryptionkey@0"
I20260619 20:31:47.590361 5756 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260619 20:31:47.591449 5771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:47.591789 5756 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:31:47.591960 5756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "bed13b3e786a4d2da073947f19d78d51"
format_stamp: "Formatted at 2026-06-19 20:31:47 on dist-test-slave-4m58"
server_key: "f7b7c4b806da18d440d01980b0a19c5d"
server_key_iv: "1e2773c9473b770daa9a6a66c0b9ea4e"
server_key_version: "encryptionkey@0"
I20260619 20:31:47.592069 5756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:31:47.606158 5756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:31:47.609076 5756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:31:47.609263 5756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:31:47.610013 5756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:31:47.610081 5756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:47.610116 5756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:31:47.610131 5756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 20:31:47 dist-test-slave-4m58 krb5kdc[5499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901107, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:31:47.618711 5756 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:34185
I20260619 20:31:47.618903 5886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:34185 every 8 connection(s)
I20260619 20:31:47.619246 5756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
I20260619 20:31:47.621138 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 5756
I20260619 20:31:47.621244 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance
I20260619 20:31:47.621423 4429 external_mini_cluster.cc:1428] Setting key dd9dee922cf032fe6afa33aa9a8bb677
I20260619 20:31:47.622763 5780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:47.610564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:38939 (local address 127.4.83.66:41743)
0619 20:31:47.610816 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:47.610899 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:47.611468 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:47.611735 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:47.611738 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:47.612114 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:31:47.612341 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:47.612347 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.613060 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.613061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:47.614113 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:47.614117 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.614554 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.614555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:47.614712 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:47.615453 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:31:47.615472 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:31:47.618806 (+ 3334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:31:47.620763 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:47.620772 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:47.620775 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:47.621118 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:47.621441 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:47.621443 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:47.621443 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:47.621607 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:47.622020 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:31:47.622029 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:31:47.622263 (+ 234us) client_negotiation.cc:769] Sending connection context
0619 20:31:47.622384 (+ 121us) client_negotiation.cc:240] Negotiation successful
0619 20:31:47.622561 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":135,"threads_started":1}
I20260619 20:31:47.622766 5593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:47.610609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:41743 (local address 127.4.83.126:38939)
0619 20:31:47.610693 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:47.610697 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:47.610998 (+ 301us) server_negotiation.cc:407] Connection header received
0619 20:31:47.611586 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:47.611588 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:47.611601 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:47.611655 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:47.612437 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.612967 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.614233 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.614476 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.614538 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:47.618950 (+ 4412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:47.618952 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:47.618952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:47.618964 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:47.620659 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:47.621226 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:47.621227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:47.621228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:47.621308 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:47.621693 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:47.621695 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:47.621695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:47.621891 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:47.621978 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:47.622432 (+ 454us) server_negotiation.cc:299] Negotiation successful
0619 20:31:47.622557 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 20:31:47.623715 5887 heartbeater.cc:347] Connected to a master server at 127.4.83.126:38939
I20260619 20:31:47.623965 5887 heartbeater.cc:464] Registering TS with master...
I20260619 20:31:47.624379 5887 heartbeater.cc:511] Master 127.4.83.126:38939 requested a full tablet report, sending...
I20260619 20:31:47.625182 5544 ts_manager.cc:194] Registered new tserver with Master: bed13b3e786a4d2da073947f19d78d51 (127.4.83.66:34185)
I20260619 20:31:47.625747 5544 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.66@KRBTEST.COM'} at 127.4.83.66:41743
WARNING: no policy specified for kudu/127.4.83.67@KRBTEST.COM; defaulting to no policy
I20260619 20:31:47.632697 5894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:47.624837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:38939 (local address 127.4.83.66:36937)
0619 20:31:47.624966 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:47.624979 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:47.625027 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:47.625244 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:47.625245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:47.625479 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:31:47.625565 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:47.625568 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.626517 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.626519 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:47.627160 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:47.627162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.627701 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.627702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:47.627778 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:47.628360 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:31:47.628365 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:31:47.628723 (+ 358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:31:47.630949 (+ 2226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:47.630950 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:47.630950 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:47.631317 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:47.631631 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:47.631632 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:47.631632 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:47.631725 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:47.632321 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:31:47.632322 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:31:47.632398 (+ 76us) client_negotiation.cc:769] Sending connection context
0619 20:31:47.632481 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 20:31:47.632586 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":74,"threads_started":1}
I20260619 20:31:47.632776 5593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:47.624943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:36937 (local address 127.4.83.126:38939)
0619 20:31:47.625003 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:47.625005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:47.625022 (+ 17us) server_negotiation.cc:407] Connection header received
0619 20:31:47.625140 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:47.625141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:47.625154 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:47.625219 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:47.625666 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.626389 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.627263 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.627591 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.627680 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:47.628836 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:47.628839 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:47.628839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:47.628850 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:47.630820 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:47.631425 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:47.631427 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:47.631427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:47.631527 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:47.631891 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:47.631893 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:47.631894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:47.632204 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:47.632310 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:47.632538 (+ 228us) server_negotiation.cc:299] Negotiation successful
0619 20:31:47.632659 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
I20260619 20:31:47.655215 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:0
--local_ip_for_outbound_sockets=127.4.83.67
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:38939
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:41271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:31:47.752427 5896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:31:47.752723 5896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:31:47.752756 5896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:31:47.754271 5896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:31:47.754348 5896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:31:47.754375 5896 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 20:31:47.754395 5896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:31:47.756363 5896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:41271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.83.126:38939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:31:47.756690 5896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:31:47.756975 5896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:31:47.763499 5902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:47.763607 5896 server_base.cc:1034] running on GCE node
W20260619 20:31:47.763511 5904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:47.763511 5901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:47.764091 5896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:31:47.764408 5896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:31:47.765630 5896 hybrid_clock.cc:648] HybridClock initialized: now 1781901107765614 us; error 35 us; skew 500 ppm
Jun 19 20:31:47 dist-test-slave-4m58 krb5kdc[5499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901107, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:31:47.768823 5896 init.cc:377] Logged in from keytab as kudu/127.4.83.67@KRBTEST.COM (short username kudu)
I20260619 20:31:47.769445 5896 webserver.cc:460] Webserver started at http://127.4.83.67:45343/ using document root <none> and password file <none>
I20260619 20:31:47.769707 5896 fs_manager.cc:359] Metadata directory not provided
I20260619 20:31:47.769758 5896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:31:47.769872 5896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:31:47.771135 5896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-2/data/instance:
uuid: "f0551fd695774352ab52824d394e36e6"
format_stamp: "Formatted at 2026-06-19 20:31:47 on dist-test-slave-4m58"
server_key: "e49e7d11ded0f381312f844c840497e6"
server_key_iv: "31938db545ef09f5cc4eb14e358089b3"
server_key_version: "encryptionkey@0"
I20260619 20:31:47.771505 5896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance:
uuid: "f0551fd695774352ab52824d394e36e6"
format_stamp: "Formatted at 2026-06-19 20:31:47 on dist-test-slave-4m58"
server_key: "e49e7d11ded0f381312f844c840497e6"
server_key_iv: "31938db545ef09f5cc4eb14e358089b3"
server_key_version: "encryptionkey@0"
I20260619 20:31:47.773247 5896 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260619 20:31:47.774354 5911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:47.774583 5896 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260619 20:31:47.774690 5896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "f0551fd695774352ab52824d394e36e6"
format_stamp: "Formatted at 2026-06-19 20:31:47 on dist-test-slave-4m58"
server_key: "e49e7d11ded0f381312f844c840497e6"
server_key_iv: "31938db545ef09f5cc4eb14e358089b3"
server_key_version: "encryptionkey@0"
I20260619 20:31:47.774799 5896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:31:47.815109 5896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:31:47.817709 5896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:31:47.817836 5896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:31:47.818359 5896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:31:47.818410 5896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:47.818439 5896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:31:47.818451 5896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:47.826395 5896 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:40865
I20260619 20:31:47.826589 6026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:40865 every 8 connection(s)
Jun 19 20:31:47 dist-test-slave-4m58 krb5kdc[5499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901107, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:31:47.826859 5896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
I20260619 20:31:47.830555 5593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:47.819117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:55785 (local address 127.4.83.126:38939)
0619 20:31:47.819268 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:47.819271 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:47.819531 (+ 260us) server_negotiation.cc:407] Connection header received
0619 20:31:47.820328 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:47.820330 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:47.820346 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:47.820419 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:47.821605 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.822458 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.823599 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.823948 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.824028 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:47.827363 (+ 3335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:47.827365 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:47.827366 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:47.827377 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:47.828700 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:47.829316 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:47.829317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:47.829317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:47.829392 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:47.829776 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:47.829777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:47.829777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:47.829917 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:47.829995 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:47.830393 (+ 398us) server_negotiation.cc:299] Negotiation successful
0619 20:31:47.830482 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 20:31:47.830933 5920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:47.819114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:38939 (local address 127.4.83.67:55785)
0619 20:31:47.819357 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:47.819406 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:47.820154 (+ 748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:47.820532 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:47.820537 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:47.820858 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:31:47.821474 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:47.821483 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.822552 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.822553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:47.823501 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:47.823503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.824088 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.824091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:47.824356 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:47.825161 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:31:47.825174 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:31:47.827205 (+ 2031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:31:47.828838 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:47.828847 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:47.828850 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:47.829198 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:47.829501 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:47.829502 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:47.829502 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:47.829642 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:47.830016 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:31:47.830023 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:31:47.830209 (+ 186us) client_negotiation.cc:769] Sending connection context
0619 20:31:47.830331 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 20:31:47.830645 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":135,"threads_started":1}
I20260619 20:31:47.831606 6027 heartbeater.cc:347] Connected to a master server at 127.4.83.126:38939
I20260619 20:31:47.831729 6027 heartbeater.cc:464] Registering TS with master...
I20260619 20:31:47.832006 6027 heartbeater.cc:511] Master 127.4.83.126:38939 requested a full tablet report, sending...
I20260619 20:31:47.832337 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 5896
I20260619 20:31:47.832429 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance
I20260619 20:31:47.832587 4429 external_mini_cluster.cc:1428] Setting key ceb4573bf4fad9ab1b05ae66ae2ebdcc
I20260619 20:31:47.832578 5544 ts_manager.cc:194] Registered new tserver with Master: f0551fd695774352ab52824d394e36e6 (127.4.83.67:40865)
I20260619 20:31:47.833148 5544 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.67@KRBTEST.COM'} at 127.4.83.67:55785
I20260619 20:31:47.833635 4429 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:31:47.840811 5593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:47.833036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:55133 (local address 127.4.83.126:38939)
0619 20:31:47.833082 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:47.833084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:47.833317 (+ 233us) server_negotiation.cc:407] Connection header received
0619 20:31:47.833395 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:47.833396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:47.833409 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:47.833472 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:47.833947 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.834655 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.835524 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.835973 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.836048 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:47.837192 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:47.837194 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:47.837194 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:47.837201 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:47.839073 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:47.839646 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:47.839649 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:47.839649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:47.839748 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:47.840072 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:47.840073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:47.840073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:47.840256 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:47.840342 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:47.840594 (+ 252us) server_negotiation.cc:299] Negotiation successful
0619 20:31:47.840701 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:31:47.840870 6033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:47.832997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:38939 (local address 127.4.83.67:55133)
0619 20:31:47.833235 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:47.833251 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:47.833330 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:47.833477 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:47.833478 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:47.833755 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:31:47.833854 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:47.833857 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.834810 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.834811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:47.835389 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:47.835391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.836053 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.836054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:47.836150 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:47.836715 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:31:47.836720 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:31:47.837086 (+ 366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:31:47.839227 (+ 2141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:47.839230 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:47.839231 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:47.839481 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:47.839879 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:47.839881 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:47.839882 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:47.839960 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:47.840390 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:31:47.840391 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:31:47.840467 (+ 76us) client_negotiation.cc:769] Sending connection context
0619 20:31:47.840552 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 20:31:47.840704 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":153,"threads_started":1}
Jun 19 20:31:47 dist-test-slave-4m58 krb5kdc[5499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901107, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 20:31:47 dist-test-slave-4m58 krb5kdc[5499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901107, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:31:47.861285 5593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:47.852625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56666 (local address 127.4.83.126:38939)
0619 20:31:47.852698 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:47.852700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:47.852710 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:31:47.852799 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:47.852800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:47.852814 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:47.852862 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:47.853505 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.854054 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.854879 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.855067 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.855130 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:47.858280 (+ 3150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:47.858282 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:47.858283 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:47.858294 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:47.860118 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:47.860532 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:47.860533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:47.860534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:47.860589 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:47.860752 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:47.860753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:47.860753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:47.860879 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:47.860964 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:47.861090 (+ 126us) server_negotiation.cc:299] Negotiation successful
0619 20:31:47.861175 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 20:31:47.864493 5543 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56666:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:31:47.866343 5543 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:31:47.874639 6046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:47.869435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:34491 (local address 127.0.0.1:44886)
0619 20:31:47.869831 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:47.869839 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:47.869879 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:47.870266 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:47.870271 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:47.870274 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:31:47.870392 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:47.870395 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.871515 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.871517 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:47.873205 (+ 1688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:47.873207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.874320 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.874322 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:47.874404 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:47.874410 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 20:31:47.874458 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 20:31:47.874560 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":1,"thread_start_us":115,"threads_started":1}
I20260619 20:31:47.874645 6047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:47.869663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44886 (local address 127.4.83.65:34491)
0619 20:31:47.870022 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:47.870029 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:47.870043 (+ 14us) server_negotiation.cc:407] Connection header received
0619 20:31:47.870093 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:47.870095 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:47.870115 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:47.870472 (+ 357us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:31:47.870537 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.871394 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.873311 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.874220 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.874305 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:47.874360 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:47.874483 (+ 123us) server_negotiation.cc:299] Negotiation successful
0619 20:31:47.874529 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":80,"threads_started":1}
I20260619 20:31:47.874810 6044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:47.869432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:40865 (local address 127.0.0.1:47298)
0619 20:31:47.869655 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:47.869700 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:47.869776 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:47.870296 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:47.870297 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:47.870299 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:31:47.870411 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:47.870414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.871806 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.871807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:47.872972 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:47.872976 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.874372 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.874374 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:47.874542 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:47.874550 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 20:31:47.874622 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 20:31:47.874687 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":3,"thread_start_us":89,"threads_started":1}
I20260619 20:31:47.875095 6049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:47.869757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47298 (local address 127.4.83.67:40865)
0619 20:31:47.870127 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:47.870131 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:47.870141 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:31:47.870182 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:47.870184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:47.870199 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:47.870503 (+ 304us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:31:47.870577 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.871720 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.873085 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.873890 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.874131 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:47.874217 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:47.874937 (+ 720us) server_negotiation.cc:299] Negotiation successful
0619 20:31:47.874990 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":119,"threads_started":1}
I20260619 20:31:47.875118 6045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:47.869459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:34185 (local address 127.0.0.1:54226)
0619 20:31:47.869943 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:47.869950 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:47.869983 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:47.870276 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:47.870278 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:47.870281 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:31:47.870410 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:47.870414 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.872067 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.872069 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:47.873059 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:47.873064 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.873929 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.873930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:47.874033 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:47.874341 (+ 308us) client_negotiation.cc:769] Sending connection context
0619 20:31:47.874994 (+ 653us) client_negotiation.cc:240] Negotiation successful
0619 20:31:47.875057 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":40,"thread_start_us":94,"threads_started":1}
I20260619 20:31:47.875267 6048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:47.869757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54226 (local address 127.4.83.66:34185)
0619 20:31:47.870020 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:47.870025 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:47.870036 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:31:47.870077 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:47.870080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:47.870096 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:47.870171 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:31:47.870494 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.871702 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.873188 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:47.873828 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:47.873905 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:47.873984 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:47.875048 (+ 1064us) server_negotiation.cc:299] Negotiation successful
0619 20:31:47.875117 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":116,"threads_started":1}
I20260619 20:31:47.875886 5961 tablet_service.cc:1467] Processing CreateTablet for tablet 1ea679561b034981860a014a87292238 (DEFAULT_TABLE table=test-table [id=079a98459ca344629e4f600675e42a68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:31:47.876293 5961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ea679561b034981860a014a87292238. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:47.876250 5821 tablet_service.cc:1467] Processing CreateTablet for tablet 1ea679561b034981860a014a87292238 (DEFAULT_TABLE table=test-table [id=079a98459ca344629e4f600675e42a68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:31:47.876592 5821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ea679561b034981860a014a87292238. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:47.876600 5681 tablet_service.cc:1467] Processing CreateTablet for tablet 1ea679561b034981860a014a87292238 (DEFAULT_TABLE table=test-table [id=079a98459ca344629e4f600675e42a68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:31:47.877019 5681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ea679561b034981860a014a87292238. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:47.880543 6050 tablet_bootstrap.cc:492] T 1ea679561b034981860a014a87292238 P f0551fd695774352ab52824d394e36e6: Bootstrap starting.
I20260619 20:31:47.881541 6050 tablet_bootstrap.cc:654] T 1ea679561b034981860a014a87292238 P f0551fd695774352ab52824d394e36e6: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:47.881870 6050 log.cc:824] T 1ea679561b034981860a014a87292238 P f0551fd695774352ab52824d394e36e6: Log is configured to *not* fsync() on all Append() calls
I20260619 20:31:47.882259 6051 tablet_bootstrap.cc:492] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51: Bootstrap starting.
I20260619 20:31:47.882735 6050 tablet_bootstrap.cc:492] T 1ea679561b034981860a014a87292238 P f0551fd695774352ab52824d394e36e6: No bootstrap required, opened a new log
I20260619 20:31:47.882826 6050 ts_tablet_manager.cc:1397] T 1ea679561b034981860a014a87292238 P f0551fd695774352ab52824d394e36e6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:31:47.882970 6051 tablet_bootstrap.cc:654] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:47.883301 6051 log.cc:824] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51: Log is configured to *not* fsync() on all Append() calls
I20260619 20:31:47.884124 6051 tablet_bootstrap.cc:492] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51: No bootstrap required, opened a new log
I20260619 20:31:47.885732 6051 ts_tablet_manager.cc:1397] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 20:31:47.887439 6054 tablet_bootstrap.cc:492] T 1ea679561b034981860a014a87292238 P 9a028fe2ca8b4236a5d5f66dca0db482: Bootstrap starting.
I20260619 20:31:47.888648 6054 tablet_bootstrap.cc:654] T 1ea679561b034981860a014a87292238 P 9a028fe2ca8b4236a5d5f66dca0db482: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:47.889030 6054 log.cc:824] T 1ea679561b034981860a014a87292238 P 9a028fe2ca8b4236a5d5f66dca0db482: Log is configured to *not* fsync() on all Append() calls
I20260619 20:31:47.889963 6054 tablet_bootstrap.cc:492] T 1ea679561b034981860a014a87292238 P 9a028fe2ca8b4236a5d5f66dca0db482: No bootstrap required, opened a new log
I20260619 20:31:47.890048 6054 ts_tablet_manager.cc:1397] T 1ea679561b034981860a014a87292238 P 9a028fe2ca8b4236a5d5f66dca0db482: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 20:31:47.892261 6054 raft_consensus.cc:348] T 1ea679561b034981860a014a87292238 P 9a028fe2ca8b4236a5d5f66dca0db482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } } peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } }
I20260619 20:31:47.892563 6054 raft_consensus.cc:374] T 1ea679561b034981860a014a87292238 P 9a028fe2ca8b4236a5d5f66dca0db482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:47.892609 6054 raft_consensus.cc:729] T 1ea679561b034981860a014a87292238 P 9a028fe2ca8b4236a5d5f66dca0db482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a028fe2ca8b4236a5d5f66dca0db482, State: Initialized, Role: FOLLOWER
I20260619 20:31:47.892539 6050 raft_consensus.cc:348] T 1ea679561b034981860a014a87292238 P f0551fd695774352ab52824d394e36e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } } peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } }
I20260619 20:31:47.892808 6050 raft_consensus.cc:374] T 1ea679561b034981860a014a87292238 P f0551fd695774352ab52824d394e36e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:47.892853 6050 raft_consensus.cc:729] T 1ea679561b034981860a014a87292238 P f0551fd695774352ab52824d394e36e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0551fd695774352ab52824d394e36e6, State: Initialized, Role: FOLLOWER
I20260619 20:31:47.892833 6054 consensus_queue.cc:260] T 1ea679561b034981860a014a87292238 P 9a028fe2ca8b4236a5d5f66dca0db482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } } peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } }
I20260619 20:31:47.893060 6050 consensus_queue.cc:260] T 1ea679561b034981860a014a87292238 P f0551fd695774352ab52824d394e36e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } } peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } }
I20260619 20:31:47.893201 6054 ts_tablet_manager.cc:1428] T 1ea679561b034981860a014a87292238 P 9a028fe2ca8b4236a5d5f66dca0db482: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:31:47.893368 5747 heartbeater.cc:503] Master 127.4.83.126:38939 was elected leader, sending a full tablet report...
I20260619 20:31:47.894140 6050 ts_tablet_manager.cc:1428] T 1ea679561b034981860a014a87292238 P f0551fd695774352ab52824d394e36e6: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260619 20:31:47.894383 6027 heartbeater.cc:503] Master 127.4.83.126:38939 was elected leader, sending a full tablet report...
I20260619 20:31:47.895150 6051 raft_consensus.cc:348] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } } peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } }
I20260619 20:31:47.895512 6051 raft_consensus.cc:374] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:47.895560 6051 raft_consensus.cc:729] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bed13b3e786a4d2da073947f19d78d51, State: Initialized, Role: FOLLOWER
I20260619 20:31:47.895721 6051 consensus_queue.cc:260] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } } peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } }
I20260619 20:31:47.896092 5887 heartbeater.cc:503] Master 127.4.83.126:38939 was elected leader, sending a full tablet report...
I20260619 20:31:47.896111 6051 ts_tablet_manager.cc:1428] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
W20260619 20:31:47.926734 5748 tablet.cc:2369] T 1ea679561b034981860a014a87292238 P 9a028fe2ca8b4236a5d5f66dca0db482: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:31:48.077672 6028 tablet.cc:2369] T 1ea679561b034981860a014a87292238 P f0551fd695774352ab52824d394e36e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:31:48.120275 5888 tablet.cc:2369] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:31:48.155918 6058 raft_consensus.cc:482] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:31:48.156108 6058 raft_consensus.cc:504] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } } peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } }
I20260619 20:31:48.156625 6058 leader_election.cc:288] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0551fd695774352ab52824d394e36e6 (127.4.83.67:40865), 9a028fe2ca8b4236a5d5f66dca0db482 (127.4.83.65:34491)
I20260619 20:31:48.160058 6059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:48.156929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:34491 (local address 127.4.83.66:44129)
0619 20:31:48.157193 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:48.157206 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:48.157293 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:48.157500 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:48.157501 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:48.157504 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:31:48.157604 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:48.157606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.158324 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.158325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:48.159215 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:48.159217 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.159752 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.159753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:48.159854 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:48.159859 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:31:48.159916 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 20:31:48.159965 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":88,"threads_started":1}
I20260619 20:31:48.160046 6047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:48.157019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:44129 (local address 127.4.83.65:34491)
0619 20:31:48.157188 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:48.157190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:48.157279 (+ 89us) server_negotiation.cc:407] Connection header received
0619 20:31:48.157373 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:48.157375 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:48.157390 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:48.157474 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:31:48.157666 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.158158 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.159296 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.159672 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.159738 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:48.159750 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:48.159923 (+ 173us) server_negotiation.cc:299] Negotiation successful
0619 20:31:48.159968 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260619 20:31:48.160398 5701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ea679561b034981860a014a87292238" candidate_uuid: "bed13b3e786a4d2da073947f19d78d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" is_pre_election: true
I20260619 20:31:48.160583 5701 raft_consensus.cc:2454] T 1ea679561b034981860a014a87292238 P 9a028fe2ca8b4236a5d5f66dca0db482 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bed13b3e786a4d2da073947f19d78d51 in term 0.
I20260619 20:31:48.160660 6049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:48.157029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:43181 (local address 127.4.83.67:40865)
0619 20:31:48.157214 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:48.157216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:48.157298 (+ 82us) server_negotiation.cc:407] Connection header received
0619 20:31:48.157382 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:48.157383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:48.157395 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:48.157445 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:31:48.157636 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.158586 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.159753 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.160269 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.160351 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:48.160364 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:48.160506 (+ 142us) server_negotiation.cc:299] Negotiation successful
0619 20:31:48.160568 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
I20260619 20:31:48.160686 6060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:48.156879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:40865 (local address 127.4.83.66:43181)
0619 20:31:48.157181 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:48.157202 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:48.157299 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:48.157459 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:48.157460 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:48.157463 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:31:48.157567 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:48.157570 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.158704 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.158705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:48.159627 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:48.159633 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.160358 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.160359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:48.160442 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:48.160446 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:31:48.160494 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 20:31:48.160604 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":117,"threads_started":1}
I20260619 20:31:48.160849 5773 leader_election.cc:302] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a028fe2ca8b4236a5d5f66dca0db482, bed13b3e786a4d2da073947f19d78d51; no voters:
I20260619 20:31:48.161038 6058 raft_consensus.cc:2781] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:31:48.161096 6058 raft_consensus.cc:482] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:31:48.161024 5981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ea679561b034981860a014a87292238" candidate_uuid: "bed13b3e786a4d2da073947f19d78d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0551fd695774352ab52824d394e36e6" is_pre_election: true
I20260619 20:31:48.161111 6058 raft_consensus.cc:3037] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:48.161293 5981 raft_consensus.cc:2454] T 1ea679561b034981860a014a87292238 P f0551fd695774352ab52824d394e36e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bed13b3e786a4d2da073947f19d78d51 in term 0.
I20260619 20:31:48.161903 6058 raft_consensus.cc:504] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } } peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } }
I20260619 20:31:48.162068 6058 leader_election.cc:288] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51 [CANDIDATE]: Term 1 election: Requested vote from peers f0551fd695774352ab52824d394e36e6 (127.4.83.67:40865), 9a028fe2ca8b4236a5d5f66dca0db482 (127.4.83.65:34491)
I20260619 20:31:48.162240 5701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ea679561b034981860a014a87292238" candidate_uuid: "bed13b3e786a4d2da073947f19d78d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a028fe2ca8b4236a5d5f66dca0db482"
I20260619 20:31:48.162323 5701 raft_consensus.cc:3037] T 1ea679561b034981860a014a87292238 P 9a028fe2ca8b4236a5d5f66dca0db482 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:48.162379 5981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ea679561b034981860a014a87292238" candidate_uuid: "bed13b3e786a4d2da073947f19d78d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0551fd695774352ab52824d394e36e6"
I20260619 20:31:48.162472 5981 raft_consensus.cc:3037] T 1ea679561b034981860a014a87292238 P f0551fd695774352ab52824d394e36e6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:48.163307 5701 raft_consensus.cc:2454] T 1ea679561b034981860a014a87292238 P 9a028fe2ca8b4236a5d5f66dca0db482 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bed13b3e786a4d2da073947f19d78d51 in term 1.
I20260619 20:31:48.163307 5981 raft_consensus.cc:2454] T 1ea679561b034981860a014a87292238 P f0551fd695774352ab52824d394e36e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bed13b3e786a4d2da073947f19d78d51 in term 1.
I20260619 20:31:48.163528 5773 leader_election.cc:302] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a028fe2ca8b4236a5d5f66dca0db482, bed13b3e786a4d2da073947f19d78d51; no voters:
I20260619 20:31:48.163646 6058 raft_consensus.cc:2781] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:31:48.163766 6058 raft_consensus.cc:686] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51 [term 1 LEADER]: Becoming Leader. State: Replica: bed13b3e786a4d2da073947f19d78d51, State: Running, Role: LEADER
I20260619 20:31:48.163904 6058 consensus_queue.cc:237] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } } peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } }
I20260619 20:31:48.164775 5544 catalog_manager.cc:5495] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51 reported cstate change: term changed from 0 to 1, leader changed from <none> to bed13b3e786a4d2da073947f19d78d51 (127.4.83.66). New cstate: current_term: 1 leader_uuid: "bed13b3e786a4d2da073947f19d78d51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } health_report { overall_health: UNKNOWN } } }
I20260619 20:31:48.240715 6048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:48.237752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54236 (local address 127.4.83.66:34185)
0619 20:31:48.237816 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:48.237819 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:48.237828 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:31:48.237987 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:48.237988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:48.238029 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:48.238106 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:31:48.238367 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.238923 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.239821 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.240071 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.240136 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:48.240350 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:31:48.240472 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:31:48.240548 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:48.240588 (+ 40us) server_negotiation.cc:299] Negotiation successful
0619 20:31:48.240608 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 20:31:48.243067 5701 raft_consensus.cc:1261] T 1ea679561b034981860a014a87292238 P 9a028fe2ca8b4236a5d5f66dca0db482 [term 1 FOLLOWER]: Refusing update from remote peer bed13b3e786a4d2da073947f19d78d51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:31:48.243140 5981 raft_consensus.cc:1261] T 1ea679561b034981860a014a87292238 P f0551fd695774352ab52824d394e36e6 [term 1 FOLLOWER]: Refusing update from remote peer bed13b3e786a4d2da073947f19d78d51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:31:48.243551 6061 consensus_queue.cc:1035] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:31:48.243752 6058 consensus_queue.cc:1035] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:31:48.246639 6066 mvcc.cc:205] Tried to move back new op lower bound from 7298666939360059392 to 7298666939040210944. Current Snapshot: MvccSnapshot[applied={T|T < 7298666939360059392}]
I20260619 20:31:48.246827 6064 mvcc.cc:205] Tried to move back new op lower bound from 7298666939360059392 to 7298666939040210944. Current Snapshot: MvccSnapshot[applied={T|T < 7298666939360059392}]
I20260619 20:31:48.247121 6065 mvcc.cc:205] Tried to move back new op lower bound from 7298666939360059392 to 7298666939040210944. Current Snapshot: MvccSnapshot[applied={T|T < 7298666939360059392}]
I20260619 20:31:48.250942 5544 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:31:48.251045 5544 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:31:48.251936 5544 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 11f488cd7e054da6ae03786d0972c1f6: Sending DeleteTablet for 3 replicas of tablet 1ea679561b034981860a014a87292238
I20260619 20:31:48.252272 5821 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ea679561b034981860a014a87292238 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:31:48 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:54226
I20260619 20:31:48.252336 5961 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ea679561b034981860a014a87292238 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:31:48 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:47298
I20260619 20:31:48.252655 5681 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ea679561b034981860a014a87292238 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:31:48 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:44886
I20260619 20:31:48.252910 6075 tablet_replica.cc:330] T 1ea679561b034981860a014a87292238 P 9a028fe2ca8b4236a5d5f66dca0db482: stopping tablet replica
I20260619 20:31:48.252904 6074 tablet_replica.cc:330] T 1ea679561b034981860a014a87292238 P f0551fd695774352ab52824d394e36e6: stopping tablet replica
I20260619 20:31:48.252892 5544 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56666:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:31:48.253039 5544 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:31:48.253038 6074 raft_consensus.cc:2229] T 1ea679561b034981860a014a87292238 P f0551fd695774352ab52824d394e36e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:31:48.253190 6074 raft_consensus.cc:2258] T 1ea679561b034981860a014a87292238 P f0551fd695774352ab52824d394e36e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:31:48.253340 6075 raft_consensus.cc:2229] T 1ea679561b034981860a014a87292238 P 9a028fe2ca8b4236a5d5f66dca0db482 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:31:48.253490 6075 raft_consensus.cc:2258] T 1ea679561b034981860a014a87292238 P 9a028fe2ca8b4236a5d5f66dca0db482 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:31:48.254185 6075 ts_tablet_manager.cc:1905] T 1ea679561b034981860a014a87292238 P 9a028fe2ca8b4236a5d5f66dca0db482: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:31:48.254202 6074 ts_tablet_manager.cc:1905] T 1ea679561b034981860a014a87292238 P f0551fd695774352ab52824d394e36e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:31:48.254691 5681 tablet_service.cc:1467] Processing CreateTablet for tablet 4f0de9d083ce4afd9a85cc440e2a16b7 (DEFAULT_TABLE table=test-table [id=f596d3f707ec49b5a29ee2bbdacd35ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:31:48.254842 5681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f0de9d083ce4afd9a85cc440e2a16b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:48.254957 5961 tablet_service.cc:1467] Processing CreateTablet for tablet 4f0de9d083ce4afd9a85cc440e2a16b7 (DEFAULT_TABLE table=test-table [id=f596d3f707ec49b5a29ee2bbdacd35ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:31:48.255087 5961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f0de9d083ce4afd9a85cc440e2a16b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:48.256263 6054 tablet_bootstrap.cc:492] T 4f0de9d083ce4afd9a85cc440e2a16b7 P 9a028fe2ca8b4236a5d5f66dca0db482: Bootstrap starting.
I20260619 20:31:48.256273 5821 tablet_service.cc:1467] Processing CreateTablet for tablet 4f0de9d083ce4afd9a85cc440e2a16b7 (DEFAULT_TABLE table=test-table [id=f596d3f707ec49b5a29ee2bbdacd35ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:31:48.256402 5821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f0de9d083ce4afd9a85cc440e2a16b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:48.257063 6054 tablet_bootstrap.cc:654] T 4f0de9d083ce4afd9a85cc440e2a16b7 P 9a028fe2ca8b4236a5d5f66dca0db482: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:48.257329 6051 tablet_bootstrap.cc:492] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51: Bootstrap starting.
I20260619 20:31:48.257782 6051 tablet_bootstrap.cc:654] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:48.258736 6050 tablet_bootstrap.cc:492] T 4f0de9d083ce4afd9a85cc440e2a16b7 P f0551fd695774352ab52824d394e36e6: Bootstrap starting.
I20260619 20:31:48.259150 6050 tablet_bootstrap.cc:654] T 4f0de9d083ce4afd9a85cc440e2a16b7 P f0551fd695774352ab52824d394e36e6: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:48.259559 6075 ts_tablet_manager.cc:1918] T 1ea679561b034981860a014a87292238 P 9a028fe2ca8b4236a5d5f66dca0db482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:31:48.259644 6075 log.cc:1196] T 1ea679561b034981860a014a87292238 P 9a028fe2ca8b4236a5d5f66dca0db482: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-0/wal/wals/1ea679561b034981860a014a87292238
I20260619 20:31:48.259954 6075 ts_tablet_manager.cc:1939] T 1ea679561b034981860a014a87292238 P 9a028fe2ca8b4236a5d5f66dca0db482: Deleting consensus metadata
I20260619 20:31:48.260036 6074 ts_tablet_manager.cc:1918] T 1ea679561b034981860a014a87292238 P f0551fd695774352ab52824d394e36e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:31:48.260156 6074 log.cc:1196] T 1ea679561b034981860a014a87292238 P f0551fd695774352ab52824d394e36e6: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-2/wal/wals/1ea679561b034981860a014a87292238
I20260619 20:31:48.260452 6074 ts_tablet_manager.cc:1939] T 1ea679561b034981860a014a87292238 P f0551fd695774352ab52824d394e36e6: Deleting consensus metadata
I20260619 20:31:48.260460 5529 catalog_manager.cc:4841] TS 9a028fe2ca8b4236a5d5f66dca0db482 (127.4.83.65:34491): tablet 1ea679561b034981860a014a87292238 (table test-table [id=079a98459ca344629e4f600675e42a68]) successfully deleted
I20260619 20:31:48.260731 6050 tablet_bootstrap.cc:492] T 4f0de9d083ce4afd9a85cc440e2a16b7 P f0551fd695774352ab52824d394e36e6: No bootstrap required, opened a new log
I20260619 20:31:48.260780 6050 ts_tablet_manager.cc:1397] T 4f0de9d083ce4afd9a85cc440e2a16b7 P f0551fd695774352ab52824d394e36e6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:31:48.260954 6050 raft_consensus.cc:348] T 4f0de9d083ce4afd9a85cc440e2a16b7 P f0551fd695774352ab52824d394e36e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } } peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } }
I20260619 20:31:48.261046 6050 raft_consensus.cc:374] T 4f0de9d083ce4afd9a85cc440e2a16b7 P f0551fd695774352ab52824d394e36e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:48.261029 5528 catalog_manager.cc:4841] TS f0551fd695774352ab52824d394e36e6 (127.4.83.67:40865): tablet 1ea679561b034981860a014a87292238 (table test-table [id=079a98459ca344629e4f600675e42a68]) successfully deleted
I20260619 20:31:48.261066 6050 raft_consensus.cc:729] T 4f0de9d083ce4afd9a85cc440e2a16b7 P f0551fd695774352ab52824d394e36e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0551fd695774352ab52824d394e36e6, State: Initialized, Role: FOLLOWER
I20260619 20:31:48.261121 6050 consensus_queue.cc:260] T 4f0de9d083ce4afd9a85cc440e2a16b7 P f0551fd695774352ab52824d394e36e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } } peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } }
I20260619 20:31:48.261219 6050 ts_tablet_manager.cc:1428] T 4f0de9d083ce4afd9a85cc440e2a16b7 P f0551fd695774352ab52824d394e36e6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:48.263993 6073 tablet_replica.cc:330] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51: stopping tablet replica
I20260619 20:31:48.264199 6073 raft_consensus.cc:2229] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:31:48.264344 6051 tablet_bootstrap.cc:492] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51: No bootstrap required, opened a new log
I20260619 20:31:48.264355 6073 raft_consensus.cc:2258] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:31:48.264460 6054 tablet_bootstrap.cc:492] T 4f0de9d083ce4afd9a85cc440e2a16b7 P 9a028fe2ca8b4236a5d5f66dca0db482: No bootstrap required, opened a new log
I20260619 20:31:48.264520 6054 ts_tablet_manager.cc:1397] T 4f0de9d083ce4afd9a85cc440e2a16b7 P 9a028fe2ca8b4236a5d5f66dca0db482: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260619 20:31:48.264442 6051 ts_tablet_manager.cc:1397] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20260619 20:31:48.265000 6051 raft_consensus.cc:348] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } } peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } }
I20260619 20:31:48.264983 6054 raft_consensus.cc:348] T 4f0de9d083ce4afd9a85cc440e2a16b7 P 9a028fe2ca8b4236a5d5f66dca0db482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } } peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } }
I20260619 20:31:48.265074 6051 raft_consensus.cc:374] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:48.265082 6054 raft_consensus.cc:374] T 4f0de9d083ce4afd9a85cc440e2a16b7 P 9a028fe2ca8b4236a5d5f66dca0db482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:48.265092 6051 raft_consensus.cc:729] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bed13b3e786a4d2da073947f19d78d51, State: Initialized, Role: FOLLOWER
I20260619 20:31:48.265105 6054 raft_consensus.cc:729] T 4f0de9d083ce4afd9a85cc440e2a16b7 P 9a028fe2ca8b4236a5d5f66dca0db482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a028fe2ca8b4236a5d5f66dca0db482, State: Initialized, Role: FOLLOWER
I20260619 20:31:48.265147 6051 consensus_queue.cc:260] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } } peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } }
I20260619 20:31:48.265159 6054 consensus_queue.cc:260] T 4f0de9d083ce4afd9a85cc440e2a16b7 P 9a028fe2ca8b4236a5d5f66dca0db482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } } peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } }
I20260619 20:31:48.265235 6051 ts_tablet_manager.cc:1428] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:48.265290 6054 ts_tablet_manager.cc:1428] T 4f0de9d083ce4afd9a85cc440e2a16b7 P 9a028fe2ca8b4236a5d5f66dca0db482: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:31:48.266281 6073 ts_tablet_manager.cc:1905] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:31:48.266467 6058 raft_consensus.cc:482] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:31:48.266534 6058 raft_consensus.cc:504] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } } peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } }
I20260619 20:31:48.266675 6058 leader_election.cc:288] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a028fe2ca8b4236a5d5f66dca0db482 (127.4.83.65:34491), f0551fd695774352ab52824d394e36e6 (127.4.83.67:40865)
I20260619 20:31:48.266875 5701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f0de9d083ce4afd9a85cc440e2a16b7" candidate_uuid: "bed13b3e786a4d2da073947f19d78d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" is_pre_election: true
I20260619 20:31:48.266911 5981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f0de9d083ce4afd9a85cc440e2a16b7" candidate_uuid: "bed13b3e786a4d2da073947f19d78d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0551fd695774352ab52824d394e36e6" is_pre_election: true
I20260619 20:31:48.266995 5701 raft_consensus.cc:2454] T 4f0de9d083ce4afd9a85cc440e2a16b7 P 9a028fe2ca8b4236a5d5f66dca0db482 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bed13b3e786a4d2da073947f19d78d51 in term 0.
I20260619 20:31:48.266996 5981 raft_consensus.cc:2454] T 4f0de9d083ce4afd9a85cc440e2a16b7 P f0551fd695774352ab52824d394e36e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bed13b3e786a4d2da073947f19d78d51 in term 0.
I20260619 20:31:48.267170 5772 leader_election.cc:302] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bed13b3e786a4d2da073947f19d78d51, f0551fd695774352ab52824d394e36e6; no voters:
I20260619 20:31:48.267261 6058 raft_consensus.cc:2781] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:31:48.267288 6058 raft_consensus.cc:482] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:31:48.267319 6058 raft_consensus.cc:3037] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:48.268332 6058 raft_consensus.cc:504] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } } peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } }
I20260619 20:31:48.268453 6058 leader_election.cc:288] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51 [CANDIDATE]: Term 1 election: Requested vote from peers 9a028fe2ca8b4236a5d5f66dca0db482 (127.4.83.65:34491), f0551fd695774352ab52824d394e36e6 (127.4.83.67:40865)
I20260619 20:31:48.268636 5981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f0de9d083ce4afd9a85cc440e2a16b7" candidate_uuid: "bed13b3e786a4d2da073947f19d78d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0551fd695774352ab52824d394e36e6"
I20260619 20:31:48.268680 6073 ts_tablet_manager.cc:1918] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:31:48.268714 5981 raft_consensus.cc:3037] T 4f0de9d083ce4afd9a85cc440e2a16b7 P f0551fd695774352ab52824d394e36e6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:48.268687 5701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f0de9d083ce4afd9a85cc440e2a16b7" candidate_uuid: "bed13b3e786a4d2da073947f19d78d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a028fe2ca8b4236a5d5f66dca0db482"
I20260619 20:31:48.268769 6073 log.cc:1196] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-1/wal/wals/1ea679561b034981860a014a87292238
I20260619 20:31:48.268790 5701 raft_consensus.cc:3037] T 4f0de9d083ce4afd9a85cc440e2a16b7 P 9a028fe2ca8b4236a5d5f66dca0db482 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:48.269047 6073 ts_tablet_manager.cc:1939] T 1ea679561b034981860a014a87292238 P bed13b3e786a4d2da073947f19d78d51: Deleting consensus metadata
I20260619 20:31:48.269419 5981 raft_consensus.cc:2454] T 4f0de9d083ce4afd9a85cc440e2a16b7 P f0551fd695774352ab52824d394e36e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bed13b3e786a4d2da073947f19d78d51 in term 1.
I20260619 20:31:48.269619 5772 leader_election.cc:302] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bed13b3e786a4d2da073947f19d78d51, f0551fd695774352ab52824d394e36e6; no voters:
I20260619 20:31:48.269716 6058 raft_consensus.cc:2781] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:31:48.269716 5701 raft_consensus.cc:2454] T 4f0de9d083ce4afd9a85cc440e2a16b7 P 9a028fe2ca8b4236a5d5f66dca0db482 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bed13b3e786a4d2da073947f19d78d51 in term 1.
I20260619 20:31:48.269771 6058 raft_consensus.cc:686] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51 [term 1 LEADER]: Becoming Leader. State: Replica: bed13b3e786a4d2da073947f19d78d51, State: Running, Role: LEADER
I20260619 20:31:48.269836 6058 consensus_queue.cc:237] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } } peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } }
I20260619 20:31:48.269865 5531 catalog_manager.cc:4841] TS bed13b3e786a4d2da073947f19d78d51 (127.4.83.66:34185): tablet 1ea679561b034981860a014a87292238 (table test-table [id=079a98459ca344629e4f600675e42a68]) successfully deleted
I20260619 20:31:48.270371 5544 catalog_manager.cc:5495] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51 reported cstate change: term changed from 0 to 1, leader changed from <none> to bed13b3e786a4d2da073947f19d78d51 (127.4.83.66). New cstate: current_term: 1 leader_uuid: "bed13b3e786a4d2da073947f19d78d51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } health_report { overall_health: UNKNOWN } } }
Jun 19 20:31:48 dist-test-slave-4m58 krb5kdc[5499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901107, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:31:48.298002 5593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:48.289040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56682 (local address 127.4.83.126:38939)
0619 20:31:48.289126 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:48.289129 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:48.289303 (+ 174us) server_negotiation.cc:407] Connection header received
0619 20:31:48.289348 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:48.289349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:48.289363 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:48.289436 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:48.290130 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.290695 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.291938 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.292239 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.292327 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:48.295181 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:48.295183 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:48.295183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:48.295191 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:48.296830 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:48.297266 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:48.297267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:48.297267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:48.297319 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:48.297474 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:48.297474 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:48.297475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:48.297592 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:48.297647 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:48.297786 (+ 139us) server_negotiation.cc:299] Negotiation successful
0619 20:31:48.297880 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 20:31:48.298065 6085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:48.288958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:38939 (local address 127.0.0.1:56682)
0619 20:31:48.289154 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:48.289169 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:48.289292 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:48.289466 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:48.289467 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:48.289937 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:31:48.290020 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:48.290023 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.290780 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.290781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:48.291780 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:48.291782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.292338 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.292339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:48.292407 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:48.293092 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:31:48.293099 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:31:48.295083 (+ 1984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:31:48.296958 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:48.296975 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:48.296977 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:48.297199 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:48.297369 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:48.297370 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:48.297370 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:48.297424 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:48.297642 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:31:48.297651 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:31:48.297700 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 20:31:48.297759 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 20:31:48.297943 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":65,"thread_start_us":108,"threads_started":1}
I20260619 20:31:48.300906 5544 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:56682:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 20:31:48.301149 5544 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:31:48.302904 5821 tablet_service.cc:1467] Processing CreateTablet for tablet f0605cfdf3e44f19ba9814d39a780c2c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5e9675c4a070412c949b1828fb1ab4a9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 20:31:48.302984 5681 tablet_service.cc:1467] Processing CreateTablet for tablet f0605cfdf3e44f19ba9814d39a780c2c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5e9675c4a070412c949b1828fb1ab4a9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 20:31:48.303097 5681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0605cfdf3e44f19ba9814d39a780c2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:48.303098 5821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0605cfdf3e44f19ba9814d39a780c2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:48.303092 5961 tablet_service.cc:1467] Processing CreateTablet for tablet f0605cfdf3e44f19ba9814d39a780c2c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5e9675c4a070412c949b1828fb1ab4a9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 20:31:48.303311 5961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0605cfdf3e44f19ba9814d39a780c2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:48.304865 6051 tablet_bootstrap.cc:492] T f0605cfdf3e44f19ba9814d39a780c2c P bed13b3e786a4d2da073947f19d78d51: Bootstrap starting.
I20260619 20:31:48.305207 6050 tablet_bootstrap.cc:492] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6: Bootstrap starting.
I20260619 20:31:48.305284 6051 tablet_bootstrap.cc:654] T f0605cfdf3e44f19ba9814d39a780c2c P bed13b3e786a4d2da073947f19d78d51: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:48.305649 6050 tablet_bootstrap.cc:654] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:48.305908 6054 tablet_bootstrap.cc:492] T f0605cfdf3e44f19ba9814d39a780c2c P 9a028fe2ca8b4236a5d5f66dca0db482: Bootstrap starting.
I20260619 20:31:48.306295 6051 tablet_bootstrap.cc:492] T f0605cfdf3e44f19ba9814d39a780c2c P bed13b3e786a4d2da073947f19d78d51: No bootstrap required, opened a new log
I20260619 20:31:48.306325 6054 tablet_bootstrap.cc:654] T f0605cfdf3e44f19ba9814d39a780c2c P 9a028fe2ca8b4236a5d5f66dca0db482: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:48.306358 6051 ts_tablet_manager.cc:1397] T f0605cfdf3e44f19ba9814d39a780c2c P bed13b3e786a4d2da073947f19d78d51: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 20:31:48.306504 6051 raft_consensus.cc:348] T f0605cfdf3e44f19ba9814d39a780c2c P bed13b3e786a4d2da073947f19d78d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } } peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } }
I20260619 20:31:48.306574 6051 raft_consensus.cc:374] T f0605cfdf3e44f19ba9814d39a780c2c P bed13b3e786a4d2da073947f19d78d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:48.306592 6051 raft_consensus.cc:729] T f0605cfdf3e44f19ba9814d39a780c2c P bed13b3e786a4d2da073947f19d78d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bed13b3e786a4d2da073947f19d78d51, State: Initialized, Role: FOLLOWER
I20260619 20:31:48.306646 6051 consensus_queue.cc:260] T f0605cfdf3e44f19ba9814d39a780c2c P bed13b3e786a4d2da073947f19d78d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } } peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } }
I20260619 20:31:48.306756 6051 ts_tablet_manager.cc:1428] T f0605cfdf3e44f19ba9814d39a780c2c P bed13b3e786a4d2da073947f19d78d51: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:48.306927 6058 tablet_replica.cc:439] T f0605cfdf3e44f19ba9814d39a780c2c P bed13b3e786a4d2da073947f19d78d51: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } } peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } } }
I20260619 20:31:48.307027 6058 tablet_replica.cc:442] T f0605cfdf3e44f19ba9814d39a780c2c P bed13b3e786a4d2da073947f19d78d51: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:31:48.307091 6054 tablet_bootstrap.cc:492] T f0605cfdf3e44f19ba9814d39a780c2c P 9a028fe2ca8b4236a5d5f66dca0db482: No bootstrap required, opened a new log
I20260619 20:31:48.307154 6054 ts_tablet_manager.cc:1397] T f0605cfdf3e44f19ba9814d39a780c2c P 9a028fe2ca8b4236a5d5f66dca0db482: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:31:48.307355 6050 tablet_bootstrap.cc:492] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6: No bootstrap required, opened a new log
I20260619 20:31:48.307333 6054 raft_consensus.cc:348] T f0605cfdf3e44f19ba9814d39a780c2c P 9a028fe2ca8b4236a5d5f66dca0db482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } } peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } }
I20260619 20:31:48.307408 6050 ts_tablet_manager.cc:1397] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:31:48.307418 6054 raft_consensus.cc:374] T f0605cfdf3e44f19ba9814d39a780c2c P 9a028fe2ca8b4236a5d5f66dca0db482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:48.307440 6054 raft_consensus.cc:729] T f0605cfdf3e44f19ba9814d39a780c2c P 9a028fe2ca8b4236a5d5f66dca0db482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a028fe2ca8b4236a5d5f66dca0db482, State: Initialized, Role: FOLLOWER
I20260619 20:31:48.307490 6054 consensus_queue.cc:260] T f0605cfdf3e44f19ba9814d39a780c2c P 9a028fe2ca8b4236a5d5f66dca0db482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } } peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } }
I20260619 20:31:48.307600 6054 ts_tablet_manager.cc:1428] T f0605cfdf3e44f19ba9814d39a780c2c P 9a028fe2ca8b4236a5d5f66dca0db482: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 20:31:48.307566 6050 raft_consensus.cc:348] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } } peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } }
I20260619 20:31:48.307641 6050 raft_consensus.cc:374] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:48.307662 6050 raft_consensus.cc:729] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0551fd695774352ab52824d394e36e6, State: Initialized, Role: FOLLOWER
I20260619 20:31:48.307705 6050 consensus_queue.cc:260] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } } peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } }
I20260619 20:31:48.307811 6050 ts_tablet_manager.cc:1428] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:48.307894 6056 tablet_replica.cc:439] T f0605cfdf3e44f19ba9814d39a780c2c P 9a028fe2ca8b4236a5d5f66dca0db482: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } } peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } } }
I20260619 20:31:48.307991 6056 tablet_replica.cc:442] T f0605cfdf3e44f19ba9814d39a780c2c P 9a028fe2ca8b4236a5d5f66dca0db482: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:31:48.308374 6057 tablet_replica.cc:439] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } } peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } } }
I20260619 20:31:48.308493 6057 tablet_replica.cc:442] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:31:48.339551 6057 raft_consensus.cc:482] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:31:48.339663 6057 raft_consensus.cc:504] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } } peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } }
I20260619 20:31:48.340080 6057 leader_election.cc:288] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a028fe2ca8b4236a5d5f66dca0db482 (127.4.83.65:34491), bed13b3e786a4d2da073947f19d78d51 (127.4.83.66:34185)
I20260619 20:31:48.344450 6048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:48.340336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:51037 (local address 127.4.83.66:34185)
0619 20:31:48.340399 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:48.340401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:48.340550 (+ 149us) server_negotiation.cc:407] Connection header received
0619 20:31:48.340661 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:48.340662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:48.340673 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:48.340734 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:31:48.341014 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.341579 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.343245 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.343902 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.344002 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:48.344019 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:48.344296 (+ 277us) server_negotiation.cc:299] Negotiation successful
0619 20:31:48.344367 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 20:31:48.344990 6089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:48.340271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:34185 (local address 127.4.83.67:51037)
0619 20:31:48.340454 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:48.340467 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:48.340579 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:48.340782 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:48.340784 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:48.340788 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:31:48.340899 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:48.340903 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.341676 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.341677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:48.343126 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:48.343130 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.344007 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.344009 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:48.344159 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:48.344168 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 20:31:48.344830 (+ 662us) client_negotiation.cc:240] Negotiation successful
0619 20:31:48.344888 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":5,"thread_start_us":102,"threads_started":1}
I20260619 20:31:48.345082 6047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:48.340648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:52827 (local address 127.4.83.65:34491)
0619 20:31:48.340721 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:48.340723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:48.340731 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:31:48.340755 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:48.340756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:48.340766 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:48.340814 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:31:48.341061 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.341863 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.343983 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.344606 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.344679 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:48.344692 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:48.344911 (+ 219us) server_negotiation.cc:299] Negotiation successful
0619 20:31:48.344990 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 20:31:48.344990 6090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:48.340361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:34491 (local address 127.4.83.67:52827)
0619 20:31:48.340524 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:48.340536 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:48.340579 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:48.340841 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:48.340842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:48.340845 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:31:48.340946 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:48.340949 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.341971 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.341972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:48.343126 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:48.343130 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.344727 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.344728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:48.344809 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:48.344813 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:31:48.344871 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 20:31:48.344904 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":5,"thread_start_us":99,"threads_started":1}
I20260619 20:31:48.345431 5841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0605cfdf3e44f19ba9814d39a780c2c" candidate_uuid: "f0551fd695774352ab52824d394e36e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bed13b3e786a4d2da073947f19d78d51" is_pre_election: true
I20260619 20:31:48.345750 5841 raft_consensus.cc:2454] T f0605cfdf3e44f19ba9814d39a780c2c P bed13b3e786a4d2da073947f19d78d51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0551fd695774352ab52824d394e36e6 in term 0.
I20260619 20:31:48.345777 5701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0605cfdf3e44f19ba9814d39a780c2c" candidate_uuid: "f0551fd695774352ab52824d394e36e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" is_pre_election: true
I20260619 20:31:48.345908 5701 raft_consensus.cc:2454] T f0605cfdf3e44f19ba9814d39a780c2c P 9a028fe2ca8b4236a5d5f66dca0db482 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0551fd695774352ab52824d394e36e6 in term 0.
I20260619 20:31:48.346060 5915 leader_election.cc:302] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bed13b3e786a4d2da073947f19d78d51, f0551fd695774352ab52824d394e36e6; no voters:
I20260619 20:31:48.346272 6057 raft_consensus.cc:2781] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:31:48.346320 6057 raft_consensus.cc:482] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:31:48.346351 6057 raft_consensus.cc:3037] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:48.347576 6057 raft_consensus.cc:504] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } } peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } }
I20260619 20:31:48.347805 6057 leader_election.cc:288] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6 [CANDIDATE]: Term 1 election: Requested vote from peers 9a028fe2ca8b4236a5d5f66dca0db482 (127.4.83.65:34491), bed13b3e786a4d2da073947f19d78d51 (127.4.83.66:34185)
I20260619 20:31:48.348047 5701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0605cfdf3e44f19ba9814d39a780c2c" candidate_uuid: "f0551fd695774352ab52824d394e36e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a028fe2ca8b4236a5d5f66dca0db482"
I20260619 20:31:48.348152 5701 raft_consensus.cc:3037] T f0605cfdf3e44f19ba9814d39a780c2c P 9a028fe2ca8b4236a5d5f66dca0db482 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:48.348115 5841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0605cfdf3e44f19ba9814d39a780c2c" candidate_uuid: "f0551fd695774352ab52824d394e36e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bed13b3e786a4d2da073947f19d78d51"
I20260619 20:31:48.348212 5841 raft_consensus.cc:3037] T f0605cfdf3e44f19ba9814d39a780c2c P bed13b3e786a4d2da073947f19d78d51 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:48.348863 5701 raft_consensus.cc:2454] T f0605cfdf3e44f19ba9814d39a780c2c P 9a028fe2ca8b4236a5d5f66dca0db482 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0551fd695774352ab52824d394e36e6 in term 1.
I20260619 20:31:48.349047 5913 leader_election.cc:302] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a028fe2ca8b4236a5d5f66dca0db482, f0551fd695774352ab52824d394e36e6; no voters:
I20260619 20:31:48.349273 5841 raft_consensus.cc:2454] T f0605cfdf3e44f19ba9814d39a780c2c P bed13b3e786a4d2da073947f19d78d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0551fd695774352ab52824d394e36e6 in term 1.
I20260619 20:31:48.349521 6057 raft_consensus.cc:2781] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:31:48.349678 6057 raft_consensus.cc:686] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6 [term 1 LEADER]: Becoming Leader. State: Replica: f0551fd695774352ab52824d394e36e6, State: Running, Role: LEADER
I20260619 20:31:48.349792 6057 consensus_queue.cc:237] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } } peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } }
I20260619 20:31:48.350235 6091 tablet_replica.cc:439] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6: TxnStatusTablet state changed. Reason: New leader f0551fd695774352ab52824d394e36e6. Latest consensus state: current_term: 1 leader_uuid: "f0551fd695774352ab52824d394e36e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } } peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } } }
I20260619 20:31:48.350440 6091 tablet_replica.cc:442] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6: This TxnStatusTablet replica's current role is: LEADER
I20260619 20:31:48.350682 6093 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 20:31:48.350791 6093 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 20:31:48.350781 5544 catalog_manager.cc:5495] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to f0551fd695774352ab52824d394e36e6 (127.4.83.67). New cstate: current_term: 1 leader_uuid: "f0551fd695774352ab52824d394e36e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } health_report { overall_health: HEALTHY } } }
I20260619 20:31:48.362633 6085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:48.357916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:40865 (local address 127.0.0.1:47304)
0619 20:31:48.358015 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:48.358033 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:48.358108 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:48.358454 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:48.358455 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:48.358457 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:31:48.358525 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:48.358530 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.360119 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.360120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:48.361137 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:48.361141 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.361716 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.361716 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:48.361791 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:48.361823 (+ 32us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:31:48.362304 (+ 481us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:31:48.362321 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 20:31:48.362456 (+ 135us) client_negotiation.cc:240] Negotiation successful
0619 20:31:48.362528 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260619 20:31:48.362655 6049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:48.358186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47304 (local address 127.4.83.67:40865)
0619 20:31:48.358244 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:48.358246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:48.358258 (+ 12us) server_negotiation.cc:407] Connection header received
0619 20:31:48.358311 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:48.358312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:48.358338 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:48.358434 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:31:48.358649 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.359433 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.361290 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.361597 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.361715 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:48.361947 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:31:48.362127 (+ 180us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:31:48.362219 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:48.362468 (+ 249us) server_negotiation.cc:299] Negotiation successful
0619 20:31:48.362512 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 20:31:48.364351 5701 raft_consensus.cc:1261] T f0605cfdf3e44f19ba9814d39a780c2c P 9a028fe2ca8b4236a5d5f66dca0db482 [term 1 FOLLOWER]: Refusing update from remote peer f0551fd695774352ab52824d394e36e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:31:48.364516 5841 raft_consensus.cc:1261] T f0605cfdf3e44f19ba9814d39a780c2c P bed13b3e786a4d2da073947f19d78d51 [term 1 FOLLOWER]: Refusing update from remote peer f0551fd695774352ab52824d394e36e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:31:48.364940 6057 consensus_queue.cc:1035] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:31:48.365226 6091 consensus_queue.cc:1035] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:31:48.366426 6056 tablet_replica.cc:439] T f0605cfdf3e44f19ba9814d39a780c2c P 9a028fe2ca8b4236a5d5f66dca0db482: TxnStatusTablet state changed. Reason: New leader f0551fd695774352ab52824d394e36e6. Latest consensus state: current_term: 1 leader_uuid: "f0551fd695774352ab52824d394e36e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } } peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } } }
I20260619 20:31:48.366542 6056 tablet_replica.cc:442] T f0605cfdf3e44f19ba9814d39a780c2c P 9a028fe2ca8b4236a5d5f66dca0db482: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:31:48.368052 6058 tablet_replica.cc:439] T f0605cfdf3e44f19ba9814d39a780c2c P bed13b3e786a4d2da073947f19d78d51: TxnStatusTablet state changed. Reason: New leader f0551fd695774352ab52824d394e36e6. Latest consensus state: current_term: 1 leader_uuid: "f0551fd695774352ab52824d394e36e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } } peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } } }
I20260619 20:31:48.368191 6058 tablet_replica.cc:442] T f0605cfdf3e44f19ba9814d39a780c2c P bed13b3e786a4d2da073947f19d78d51: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:31:48.368239 6057 tablet_replica.cc:439] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f0551fd695774352ab52824d394e36e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } } peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } } }
I20260619 20:31:48.368353 6057 tablet_replica.cc:442] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6: This TxnStatusTablet replica's current role is: LEADER
I20260619 20:31:48.370314 6091 tablet_replica.cc:439] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f0551fd695774352ab52824d394e36e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } } peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } } }
I20260619 20:31:48.370438 6091 tablet_replica.cc:442] T f0605cfdf3e44f19ba9814d39a780c2c P f0551fd695774352ab52824d394e36e6: This TxnStatusTablet replica's current role is: LEADER
I20260619 20:31:48.371343 6056 tablet_replica.cc:439] T f0605cfdf3e44f19ba9814d39a780c2c P 9a028fe2ca8b4236a5d5f66dca0db482: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f0551fd695774352ab52824d394e36e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } } peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } } }
I20260619 20:31:48.371485 6056 tablet_replica.cc:442] T f0605cfdf3e44f19ba9814d39a780c2c P 9a028fe2ca8b4236a5d5f66dca0db482: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:31:48.372287 6058 tablet_replica.cc:439] T f0605cfdf3e44f19ba9814d39a780c2c P bed13b3e786a4d2da073947f19d78d51: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f0551fd695774352ab52824d394e36e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 } } peers { permanent_uuid: "bed13b3e786a4d2da073947f19d78d51" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34185 } } peers { permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 } } }
I20260619 20:31:48.372412 6058 tablet_replica.cc:442] T f0605cfdf3e44f19ba9814d39a780c2c P bed13b3e786a4d2da073947f19d78d51: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:31:48.383363 6049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:48.378686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:41117 (local address 127.4.83.67:40865)
0619 20:31:48.378750 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:48.378752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:48.378799 (+ 47us) server_negotiation.cc:407] Connection header received
0619 20:31:48.379039 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:48.379040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:48.379053 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:48.379152 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:31:48.379375 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.380201 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.382321 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.382641 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.382751 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:48.383051 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:31:48.383096 (+ 45us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:31:48.383161 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:48.383241 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 20:31:48.383268 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 20:31:48.383901 6100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:48.378517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:40865 (local address 127.4.83.66:41117)
0619 20:31:48.378696 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:48.378712 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:48.378822 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:48.379176 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:48.379177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:48.379181 (+ 4us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:31:48.379294 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:48.379297 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.380312 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.380313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:48.382014 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:48.382021 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.382815 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.382817 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:48.382914 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:48.382939 (+ 25us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:31:48.383171 (+ 232us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:31:48.383173 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:31:48.383324 (+ 151us) client_negotiation.cc:240] Negotiation successful
0619 20:31:48.383408 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":83,"threads_started":1}
I20260619 20:31:48.387768 5701 raft_consensus.cc:1261] T 4f0de9d083ce4afd9a85cc440e2a16b7 P 9a028fe2ca8b4236a5d5f66dca0db482 [term 1 FOLLOWER]: Refusing update from remote peer bed13b3e786a4d2da073947f19d78d51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:31:48.387773 5981 raft_consensus.cc:1261] T 4f0de9d083ce4afd9a85cc440e2a16b7 P f0551fd695774352ab52824d394e36e6 [term 1 FOLLOWER]: Refusing update from remote peer bed13b3e786a4d2da073947f19d78d51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:31:48.388441 6061 consensus_queue.cc:1035] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a028fe2ca8b4236a5d5f66dca0db482" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:31:48.388621 6058 consensus_queue.cc:1035] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0551fd695774352ab52824d394e36e6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:31:48.401659 6048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:48.397738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:42945 (local address 127.4.83.66:34185)
0619 20:31:48.397784 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:48.397786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:48.397938 (+ 152us) server_negotiation.cc:407] Connection header received
0619 20:31:48.398027 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:48.398028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:48.398044 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:48.398071 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:31:48.398332 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.399110 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.400562 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.400878 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.400970 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:48.401293 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:31:48.401344 (+ 51us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:31:48.401430 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:48.401535 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 20:31:48.401563 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:31:48.401722 6105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:48.397517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:34185 (local address 127.4.83.67:42945)
0619 20:31:48.397875 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:48.397886 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:48.397964 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:48.398138 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:48.398139 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:48.398141 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:31:48.398235 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:48.398237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.399217 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.399218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:48.400454 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:48.400458 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.400979 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.400980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:48.401059 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:48.401082 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:31:48.401448 (+ 366us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:31:48.401452 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:31:48.401515 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 20:31:48.401590 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":106,"threads_started":1}
I20260619 20:31:48.413758 5543 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:31:48.413918 5543 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:31:48.414882 5543 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 11f488cd7e054da6ae03786d0972c1f6: Sending DeleteTablet for 3 replicas of tablet 4f0de9d083ce4afd9a85cc440e2a16b7
I20260619 20:31:48.415179 5681 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f0de9d083ce4afd9a85cc440e2a16b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:31:48 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:44886
I20260619 20:31:48.415274 5821 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f0de9d083ce4afd9a85cc440e2a16b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:31:48 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:54226
I20260619 20:31:48.415328 6075 tablet_replica.cc:330] T 4f0de9d083ce4afd9a85cc440e2a16b7 P 9a028fe2ca8b4236a5d5f66dca0db482: stopping tablet replica
I20260619 20:31:48.415372 6073 tablet_replica.cc:330] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51: stopping tablet replica
I20260619 20:31:48.415385 6075 raft_consensus.cc:2229] T 4f0de9d083ce4afd9a85cc440e2a16b7 P 9a028fe2ca8b4236a5d5f66dca0db482 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:31:48.415426 6073 raft_consensus.cc:2229] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:31:48.415428 6075 raft_consensus.cc:2258] T 4f0de9d083ce4afd9a85cc440e2a16b7 P 9a028fe2ca8b4236a5d5f66dca0db482 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:31:48.415499 6073 raft_consensus.cc:2258] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:31:48.415673 5961 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f0de9d083ce4afd9a85cc440e2a16b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:31:48 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:47298
I20260619 20:31:48.415776 6074 tablet_replica.cc:330] T 4f0de9d083ce4afd9a85cc440e2a16b7 P f0551fd695774352ab52824d394e36e6: stopping tablet replica
I20260619 20:31:48.415863 6075 ts_tablet_manager.cc:1905] T 4f0de9d083ce4afd9a85cc440e2a16b7 P 9a028fe2ca8b4236a5d5f66dca0db482: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:31:48.415870 6074 raft_consensus.cc:2229] T 4f0de9d083ce4afd9a85cc440e2a16b7 P f0551fd695774352ab52824d394e36e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:31:48.415921 6074 raft_consensus.cc:2258] T 4f0de9d083ce4afd9a85cc440e2a16b7 P f0551fd695774352ab52824d394e36e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:31:48.416064 6073 ts_tablet_manager.cc:1905] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:31:48.416232 6074 ts_tablet_manager.cc:1905] T 4f0de9d083ce4afd9a85cc440e2a16b7 P f0551fd695774352ab52824d394e36e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:31:48.417929 6075 ts_tablet_manager.cc:1918] T 4f0de9d083ce4afd9a85cc440e2a16b7 P 9a028fe2ca8b4236a5d5f66dca0db482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:31:48.418002 6075 log.cc:1196] T 4f0de9d083ce4afd9a85cc440e2a16b7 P 9a028fe2ca8b4236a5d5f66dca0db482: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-0/wal/wals/4f0de9d083ce4afd9a85cc440e2a16b7
I20260619 20:31:48.418009 6073 ts_tablet_manager.cc:1918] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:31:48.418072 6073 log.cc:1196] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-1/wal/wals/4f0de9d083ce4afd9a85cc440e2a16b7
I20260619 20:31:48.418259 6075 ts_tablet_manager.cc:1939] T 4f0de9d083ce4afd9a85cc440e2a16b7 P 9a028fe2ca8b4236a5d5f66dca0db482: Deleting consensus metadata
I20260619 20:31:48.418319 6073 ts_tablet_manager.cc:1939] T 4f0de9d083ce4afd9a85cc440e2a16b7 P bed13b3e786a4d2da073947f19d78d51: Deleting consensus metadata
I20260619 20:31:48.418715 5529 catalog_manager.cc:4841] TS 9a028fe2ca8b4236a5d5f66dca0db482 (127.4.83.65:34491): tablet 4f0de9d083ce4afd9a85cc440e2a16b7 (table test-table [id=f596d3f707ec49b5a29ee2bbdacd35ea]) successfully deleted
I20260619 20:31:48.418787 5531 catalog_manager.cc:4841] TS bed13b3e786a4d2da073947f19d78d51 (127.4.83.66:34185): tablet 4f0de9d083ce4afd9a85cc440e2a16b7 (table test-table [id=f596d3f707ec49b5a29ee2bbdacd35ea]) successfully deleted
I20260619 20:31:48.418893 6074 ts_tablet_manager.cc:1918] T 4f0de9d083ce4afd9a85cc440e2a16b7 P f0551fd695774352ab52824d394e36e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:31:48.418977 6074 log.cc:1196] T 4f0de9d083ce4afd9a85cc440e2a16b7 P f0551fd695774352ab52824d394e36e6: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901102012824-4429-0/minicluster-data/ts-2/wal/wals/4f0de9d083ce4afd9a85cc440e2a16b7
I20260619 20:31:48.419178 6074 ts_tablet_manager.cc:1939] T 4f0de9d083ce4afd9a85cc440e2a16b7 P f0551fd695774352ab52824d394e36e6: Deleting consensus metadata
I20260619 20:31:48.419552 5528 catalog_manager.cc:4841] TS f0551fd695774352ab52824d394e36e6 (127.4.83.67:40865): tablet 4f0de9d083ce4afd9a85cc440e2a16b7 (table test-table [id=f596d3f707ec49b5a29ee2bbdacd35ea]) successfully deleted
Jun 19 20:31:48 dist-test-slave-4m58 krb5kdc[5499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901107, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.83.65@KRBTEST.COM
I20260619 20:31:48.429419 6047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:48.419138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44896 (local address 127.4.83.65:34491)
0619 20:31:48.419187 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:48.419198 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:48.419485 (+ 287us) server_negotiation.cc:407] Connection header received
0619 20:31:48.419595 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:48.419596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:48.419614 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:48.419663 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:48.420425 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.421007 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.421920 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.422139 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.422236 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:48.425801 (+ 3565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:48.425810 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:48.425811 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:48.425823 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:48.428026 (+ 2203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:48.428492 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:48.428496 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:48.428497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:48.428591 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:48.428778 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:48.428779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:48.428780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:48.428967 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:48.429116 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:48.429191 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 20:31:48.429294 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260619 20:31:48.429759 5741 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44896
I20260619 20:31:48.437446 5593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:48.430931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56692 (local address 127.4.83.126:38939)
0619 20:31:48.431007 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:48.431010 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:48.431021 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:31:48.431050 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:48.431051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:48.431063 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:48.431118 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:48.431684 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.432324 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.433026 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:48.433240 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:48.433297 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:48.434599 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:48.434600 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:48.434601 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:48.434609 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:48.436184 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:48.436559 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:48.436560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:48.436561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:48.436624 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:48.436791 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:48.436792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:48.436792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:48.436924 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:48.437019 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:48.437141 (+ 122us) server_negotiation.cc:299] Negotiation successful
0619 20:31:48.437233 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260619 20:31:48.437788 5543 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56692
I20260619 20:31:48.439884 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 5616
I20260619 20:31:48.448056 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 5756
I20260619 20:31:48.455297 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 5896
I20260619 20:31:48.462330 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 5512
2026-06-19T20:31:48Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2584 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:31:48 dist-test-slave-4m58 krb5kdc[6114](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:31:48 dist-test-slave-4m58 krb5kdc[6114](info): set up 2 sockets
Jun 19 20:31:48 dist-test-slave-4m58 krb5kdc[6114](info): commencing operation
krb5kdc: starting...
W20260619 20:31:49.444473 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.959s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:31:49 dist-test-slave-4m58 krb5kdc[6114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:31:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:31:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
I20260619 20:31:49.588953 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:44167
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:45911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:44167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:31:49.694972 6127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:31:49.695472 6127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:31:49.695518 6127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:31:49.698081 6127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:31:49.698212 6127 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 20:31:49.698235 6127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:31:49.698256 6127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:31:49.698274 6127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:31:49.698289 6127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:31:49.700678 6127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:45911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:44167
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:44167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:31:49.701084 6127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:31:49.701371 6127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:31:49.707265 6132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:49.707248 6135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:49.707265 6133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:49.707299 6127 server_base.cc:1034] running on GCE node
I20260619 20:31:49.708176 6127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:31:49.708593 6127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:31:49.709802 6127 hybrid_clock.cc:648] HybridClock initialized: now 1781901109709727 us; error 138 us; skew 500 ppm
Jun 19 20:31:49 dist-test-slave-4m58 krb5kdc[6114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901109, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:31:49.713469 6127 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:31:49.714357 6127 webserver.cc:460] Webserver started at http://127.4.83.126:42031/ using document root <none> and password file <none>
I20260619 20:31:49.714671 6127 fs_manager.cc:359] Metadata directory not provided
I20260619 20:31:49.714763 6127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:31:49.714926 6127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:31:49.716480 6127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "1164ce5a83bc48bc91a00bac7c0d9552"
format_stamp: "Formatted at 2026-06-19 20:31:49 on dist-test-slave-4m58"
server_key: "8af959559dd72782d803801b511b40e8"
server_key_iv: "a747d170dcd7c624310b6c121c31ee60"
server_key_version: "encryptionkey@0"
I20260619 20:31:49.716996 6127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "1164ce5a83bc48bc91a00bac7c0d9552"
format_stamp: "Formatted at 2026-06-19 20:31:49 on dist-test-slave-4m58"
server_key: "8af959559dd72782d803801b511b40e8"
server_key_iv: "a747d170dcd7c624310b6c121c31ee60"
server_key_version: "encryptionkey@0"
I20260619 20:31:49.718931 6127 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260619 20:31:49.720372 6142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:49.720778 6127 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:31:49.720944 6127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "1164ce5a83bc48bc91a00bac7c0d9552"
format_stamp: "Formatted at 2026-06-19 20:31:49 on dist-test-slave-4m58"
server_key: "8af959559dd72782d803801b511b40e8"
server_key_iv: "a747d170dcd7c624310b6c121c31ee60"
server_key_version: "encryptionkey@0"
I20260619 20:31:49.721055 6127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:31:49.741875 6127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:31:49.744747 6127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:31:49.744966 6127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:31:49.750114 6127 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:44167
I20260619 20:31:49.750249 6204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:44167 every 8 connection(s)
I20260619 20:31:49.750569 6127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:31:49.751513 6205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:49.754746 6205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1164ce5a83bc48bc91a00bac7c0d9552: Bootstrap starting.
I20260619 20:31:49.755789 6205 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1164ce5a83bc48bc91a00bac7c0d9552: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:49.756135 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 6127
I20260619 20:31:49.756246 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:31:49.756282 6205 log.cc:824] T 00000000000000000000000000000000 P 1164ce5a83bc48bc91a00bac7c0d9552: Log is configured to *not* fsync() on all Append() calls
I20260619 20:31:49.756450 4429 external_mini_cluster.cc:1428] Setting key a0d3737fb7fd0da8f229aa317b316ac2
I20260619 20:31:49.757658 6205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1164ce5a83bc48bc91a00bac7c0d9552: No bootstrap required, opened a new log
I20260619 20:31:49.759984 6205 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1164ce5a83bc48bc91a00bac7c0d9552 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1164ce5a83bc48bc91a00bac7c0d9552" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 44167 } }
I20260619 20:31:49.760226 6205 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1164ce5a83bc48bc91a00bac7c0d9552 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:49.760263 6205 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1164ce5a83bc48bc91a00bac7c0d9552 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1164ce5a83bc48bc91a00bac7c0d9552, State: Initialized, Role: FOLLOWER
I20260619 20:31:49.760437 6205 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1164ce5a83bc48bc91a00bac7c0d9552 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1164ce5a83bc48bc91a00bac7c0d9552" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 44167 } }
I20260619 20:31:49.760524 6205 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1164ce5a83bc48bc91a00bac7c0d9552 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:31:49.760561 6205 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1164ce5a83bc48bc91a00bac7c0d9552 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:31:49.760627 6205 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1164ce5a83bc48bc91a00bac7c0d9552 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:49.761693 6205 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1164ce5a83bc48bc91a00bac7c0d9552 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1164ce5a83bc48bc91a00bac7c0d9552" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 44167 } }
I20260619 20:31:49.761925 6205 leader_election.cc:302] T 00000000000000000000000000000000 P 1164ce5a83bc48bc91a00bac7c0d9552 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1164ce5a83bc48bc91a00bac7c0d9552; no voters:
I20260619 20:31:49.762247 6205 leader_election.cc:288] T 00000000000000000000000000000000 P 1164ce5a83bc48bc91a00bac7c0d9552 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:31:49.762300 6210 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1164ce5a83bc48bc91a00bac7c0d9552 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:31:49.762445 6210 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1164ce5a83bc48bc91a00bac7c0d9552 [term 1 LEADER]: Becoming Leader. State: Replica: 1164ce5a83bc48bc91a00bac7c0d9552, State: Running, Role: LEADER
I20260619 20:31:49.762544 6210 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1164ce5a83bc48bc91a00bac7c0d9552 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1164ce5a83bc48bc91a00bac7c0d9552" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 44167 } }
I20260619 20:31:49.762620 6205 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1164ce5a83bc48bc91a00bac7c0d9552 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:31:49.762982 6210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1164ce5a83bc48bc91a00bac7c0d9552 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1164ce5a83bc48bc91a00bac7c0d9552. Latest consensus state: current_term: 1 leader_uuid: "1164ce5a83bc48bc91a00bac7c0d9552" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1164ce5a83bc48bc91a00bac7c0d9552" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 44167 } } }
I20260619 20:31:49.763073 6210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1164ce5a83bc48bc91a00bac7c0d9552 [sys.catalog]: This master's current role is: LEADER
I20260619 20:31:49.763026 6212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1164ce5a83bc48bc91a00bac7c0d9552 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1164ce5a83bc48bc91a00bac7c0d9552" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1164ce5a83bc48bc91a00bac7c0d9552" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 44167 } } }
I20260619 20:31:49.763156 6212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1164ce5a83bc48bc91a00bac7c0d9552 [sys.catalog]: This master's current role is: LEADER
I20260619 20:31:49.763430 6215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:31:49.764068 6215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:31:49.770260 6215 catalog_manager.cc:1347] Generated new cluster ID: 10a46dd658cc420b97215ea8e6a52095
Jun 19 20:31:49 dist-test-slave-4m58 krb5kdc[6114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:31:49.770346 6215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:31:49.775318 6209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:49.757357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53480 (local address 127.4.83.126:44167)
0619 20:31:49.757619 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:49.757629 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:49.757663 (+ 34us) server_negotiation.cc:407] Connection header received
0619 20:31:49.758316 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:49.758326 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:49.758385 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:49.758525 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:49.759154 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:49.760539 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:49.761722 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:49.762284 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:49.762835 (+ 551us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:49.771087 (+ 8252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:49.771093 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:49.771093 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:49.771111 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:49.773195 (+ 2084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:49.773701 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:49.773704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:49.773705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:49.773819 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:49.774036 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:49.774037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:49.774037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:49.774367 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:49.774548 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:49.774694 (+ 146us) server_negotiation.cc:299] Negotiation successful
0619 20:31:49.774891 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":151,"threads_started":1}
I20260619 20:31:49.791404 6215 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:31:49.792189 6215 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:31:49.802479 6215 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1164ce5a83bc48bc91a00bac7c0d9552: Generated new TSK 0
I20260619 20:31:49.802888 6215 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.83.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
I20260619 20:31:49.840000 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:0
--local_ip_for_outbound_sockets=127.4.83.65
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:44167
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:45911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:31:49.934257 6231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:31:49.934492 6231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:31:49.934520 6231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:31:49.935799 6231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:31:49.935895 6231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:31:49.935927 6231 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 20:31:49.935938 6231 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 20:31:49.935956 6231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:31:49.937621 6231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:45911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.83.126:44167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:31:49.937889 6231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:31:49.938134 6231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:31:49.943581 6236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:49.943668 6239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:49.943571 6237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:49.944643 6231 server_base.cc:1034] running on GCE node
I20260619 20:31:49.944891 6231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:31:49.945147 6231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:31:49.946312 6231 hybrid_clock.cc:648] HybridClock initialized: now 1781901109946301 us; error 44 us; skew 500 ppm
Jun 19 20:31:49 dist-test-slave-4m58 krb5kdc[6114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901109, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:31:49.949110 6231 init.cc:377] Logged in from keytab as kudu/127.4.83.65@KRBTEST.COM (short username kudu)
I20260619 20:31:49.949601 6231 webserver.cc:460] Webserver started at http://127.4.83.65:38299/ using document root <none> and password file <none>
I20260619 20:31:49.949837 6231 fs_manager.cc:359] Metadata directory not provided
I20260619 20:31:49.949887 6231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:31:49.950017 6231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:31:49.951088 6231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-0/data/instance:
uuid: "e372f443fb0b481d9ca913ba70fb63c2"
format_stamp: "Formatted at 2026-06-19 20:31:49 on dist-test-slave-4m58"
server_key: "77e0ba734f4141142423154d7e85343d"
server_key_iv: "19f44c326733112840b6b288da4330b1"
server_key_version: "encryptionkey@0"
I20260619 20:31:49.951423 6231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance:
uuid: "e372f443fb0b481d9ca913ba70fb63c2"
format_stamp: "Formatted at 2026-06-19 20:31:49 on dist-test-slave-4m58"
server_key: "77e0ba734f4141142423154d7e85343d"
server_key_iv: "19f44c326733112840b6b288da4330b1"
server_key_version: "encryptionkey@0"
I20260619 20:31:49.952943 6231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:31:49.953855 6246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:49.954118 6231 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:31:49.954250 6231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "e372f443fb0b481d9ca913ba70fb63c2"
format_stamp: "Formatted at 2026-06-19 20:31:49 on dist-test-slave-4m58"
server_key: "77e0ba734f4141142423154d7e85343d"
server_key_iv: "19f44c326733112840b6b288da4330b1"
server_key_version: "encryptionkey@0"
I20260619 20:31:49.954320 6231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:31:49.976131 6231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:31:49.978446 6231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:31:49.978590 6231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:31:49.979130 6231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:31:49.979167 6231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:49.979197 6231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:31:49.979215 6231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:49.986573 6231 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:45613
I20260619 20:31:49.986963 6231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
I20260619 20:31:49.987202 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 6231
I20260619 20:31:49.987324 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance
Jun 19 20:31:49 dist-test-slave-4m58 krb5kdc[6114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901109, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:31:49.987540 4429 external_mini_cluster.cc:1428] Setting key 5dca9059656b6b3e0e093f6754af1e17
I20260619 20:31:49.986667 6361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:45613 every 8 connection(s)
I20260619 20:31:49.991753 6209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:49.980029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:33579 (local address 127.4.83.126:44167)
0619 20:31:49.980121 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:49.980124 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:49.980362 (+ 238us) server_negotiation.cc:407] Connection header received
0619 20:31:49.981079 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:49.981080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:49.981096 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:49.981147 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:49.982144 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:49.982907 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:49.984509 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:49.984906 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:49.985057 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:49.987985 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:49.987987 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:49.987987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:49.987998 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:49.989727 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:49.990473 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:49.990474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:49.990474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:49.990547 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:49.990949 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:49.990950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:49.990950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:49.991093 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:49.991170 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:49.991586 (+ 416us) server_negotiation.cc:299] Negotiation successful
0619 20:31:49.991673 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 20:31:49.992152 6255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:49.980008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:44167 (local address 127.4.83.65:33579)
0619 20:31:49.980221 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:49.980269 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:49.980972 (+ 703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:49.981243 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:49.981248 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:49.981676 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:31:49.982055 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:49.982062 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:49.983034 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:49.983036 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:49.984382 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:49.984385 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:49.985082 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:49.985083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:49.985236 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:49.985917 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:31:49.985930 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:31:49.987878 (+ 1948us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:31:49.989865 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:49.989877 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:49.989881 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:49.990287 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:49.990626 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:49.990628 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:49.990628 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:49.990824 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:49.991187 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:31:49.991194 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:31:49.991415 (+ 221us) client_negotiation.cc:769] Sending connection context
0619 20:31:49.991554 (+ 139us) client_negotiation.cc:240] Negotiation successful
0619 20:31:49.991803 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":132,"threads_started":1}
I20260619 20:31:49.992869 6362 heartbeater.cc:347] Connected to a master server at 127.4.83.126:44167
I20260619 20:31:49.992985 6362 heartbeater.cc:464] Registering TS with master...
I20260619 20:31:49.993207 6362 heartbeater.cc:511] Master 127.4.83.126:44167 requested a full tablet report, sending...
I20260619 20:31:49.993821 6159 ts_manager.cc:194] Registered new tserver with Master: e372f443fb0b481d9ca913ba70fb63c2 (127.4.83.65:45613)
I20260619 20:31:49.994488 6159 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.65@KRBTEST.COM'} at 127.4.83.65:33579
WARNING: no policy specified for kudu/127.4.83.66@KRBTEST.COM; defaulting to no policy
I20260619 20:31:50.001332 6209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:49.995057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:33951 (local address 127.4.83.126:44167)
0619 20:31:49.995258 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:49.995260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:49.995268 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:31:49.995288 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:49.995288 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:49.995299 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:49.995357 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:49.995825 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:49.996433 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:49.997132 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:49.997327 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:49.997387 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:49.998350 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:49.998351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:49.998351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:49.998359 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:50.000163 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:50.000515 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:50.000516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:50.000517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:50.000571 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:50.000769 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:50.000770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:50.000770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:50.000886 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:50.000990 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:50.001142 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 20:31:50.001228 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178}
I20260619 20:31:50.001345 6369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:49.994841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:44167 (local address 127.4.83.65:33951)
0619 20:31:49.995023 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:49.995035 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:49.995086 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:49.995406 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:49.995407 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:49.995647 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:31:49.995729 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:49.995732 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:49.996533 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:49.996534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:49.997055 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:49.997057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:49.997392 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:49.997393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:49.997474 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:49.997936 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:31:49.997941 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:31:49.998261 (+ 320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:31:50.000271 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:50.000273 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:50.000273 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:50.000451 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:50.000631 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:50.000631 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:50.000632 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:50.000688 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:50.000984 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:31:50.000986 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:31:50.001055 (+ 69us) client_negotiation.cc:769] Sending connection context
0619 20:31:50.001113 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 20:31:50.001239 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":109,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
I20260619 20:31:50.021334 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:0
--local_ip_for_outbound_sockets=127.4.83.66
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:44167
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:45911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:31:50.127272 6371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:31:50.127699 6371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:31:50.127756 6371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:31:50.130191 6371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:31:50.130332 6371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:31:50.130373 6371 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 20:31:50.130393 6371 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 20:31:50.130424 6371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:31:50.133636 6371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:45911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.83.126:44167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:31:50.134078 6371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:31:50.134446 6371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:31:50.141623 6376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:50.141624 6377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:50.141624 6379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:50.141963 6371 server_base.cc:1034] running on GCE node
I20260619 20:31:50.142184 6371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:31:50.142627 6371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:31:50.143900 6371 hybrid_clock.cc:648] HybridClock initialized: now 1781901110143868 us; error 62 us; skew 500 ppm
Jun 19 20:31:50 dist-test-slave-4m58 krb5kdc[6114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901110, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:31:50.147037 6371 init.cc:377] Logged in from keytab as kudu/127.4.83.66@KRBTEST.COM (short username kudu)
I20260619 20:31:50.147624 6371 webserver.cc:460] Webserver started at http://127.4.83.66:45399/ using document root <none> and password file <none>
I20260619 20:31:50.147914 6371 fs_manager.cc:359] Metadata directory not provided
I20260619 20:31:50.147970 6371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:31:50.148104 6371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:31:50.149643 6371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-1/data/instance:
uuid: "7482052ce1824e138d8b6e9b5b48e24d"
format_stamp: "Formatted at 2026-06-19 20:31:50 on dist-test-slave-4m58"
server_key: "4fbd4226d7ad5df85be508fb169940ff"
server_key_iv: "d156f03bd186609be451c3e0710f1cee"
server_key_version: "encryptionkey@0"
I20260619 20:31:50.150290 6371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance:
uuid: "7482052ce1824e138d8b6e9b5b48e24d"
format_stamp: "Formatted at 2026-06-19 20:31:50 on dist-test-slave-4m58"
server_key: "4fbd4226d7ad5df85be508fb169940ff"
server_key_iv: "d156f03bd186609be451c3e0710f1cee"
server_key_version: "encryptionkey@0"
I20260619 20:31:50.152251 6371 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260619 20:31:50.153327 6386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:50.153577 6371 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:31:50.153690 6371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "7482052ce1824e138d8b6e9b5b48e24d"
format_stamp: "Formatted at 2026-06-19 20:31:50 on dist-test-slave-4m58"
server_key: "4fbd4226d7ad5df85be508fb169940ff"
server_key_iv: "d156f03bd186609be451c3e0710f1cee"
server_key_version: "encryptionkey@0"
I20260619 20:31:50.153796 6371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:31:50.180743 6371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:31:50.183440 6371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:31:50.183568 6371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:31:50.184259 6371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:31:50.184314 6371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:50.184340 6371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:31:50.184353 6371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:50.192400 6371 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:40339
I20260619 20:31:50.192710 6501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:40339 every 8 connection(s)
I20260619 20:31:50.192888 6371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
Jun 19 20:31:50 dist-test-slave-4m58 krb5kdc[6114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901110, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:31:50.197844 6209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:50.185185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:33469 (local address 127.4.83.126:44167)
0619 20:31:50.185276 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:50.185279 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:50.185586 (+ 307us) server_negotiation.cc:407] Connection header received
0619 20:31:50.186468 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:50.186470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:50.186490 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:50.186587 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:50.187669 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.188552 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.190272 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.190517 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.190615 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:50.194117 (+ 3502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:50.194119 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:50.194120 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:50.194135 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:50.195823 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:50.196457 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:50.196459 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:50.196460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:50.196548 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:50.196998 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:50.196999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:50.196999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:50.197199 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:50.197284 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:50.197654 (+ 370us) server_negotiation.cc:299] Negotiation successful
0619 20:31:50.197772 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 20:31:50.197916 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:50.185148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:44167 (local address 127.4.83.66:33469)
0619 20:31:50.185385 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:50.185441 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:50.186329 (+ 888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:50.186742 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:50.186750 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:50.187159 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:31:50.187506 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:50.187513 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.188715 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.188717 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:50.190091 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:50.190095 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.190645 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.190646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:50.190831 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:50.191920 (+ 1089us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:31:50.191930 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:31:50.193934 (+ 2004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:31:50.195978 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:50.195989 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:50.195992 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:50.196320 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:50.196738 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:50.196739 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:50.196739 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:50.196890 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:50.197300 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:31:50.197304 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:31:50.197454 (+ 150us) client_negotiation.cc:769] Sending connection context
0619 20:31:50.197554 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 20:31:50.197708 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":143,"threads_started":1}
I20260619 20:31:50.198661 6502 heartbeater.cc:347] Connected to a master server at 127.4.83.126:44167
I20260619 20:31:50.198771 6502 heartbeater.cc:464] Registering TS with master...
I20260619 20:31:50.199023 6502 heartbeater.cc:511] Master 127.4.83.126:44167 requested a full tablet report, sending...
I20260619 20:31:50.199092 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 6371
I20260619 20:31:50.199213 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance
I20260619 20:31:50.199441 4429 external_mini_cluster.cc:1428] Setting key 6597680cfd8777d271cf22d13cb36ad5
I20260619 20:31:50.199519 6158 ts_manager.cc:194] Registered new tserver with Master: 7482052ce1824e138d8b6e9b5b48e24d (127.4.83.66:40339)
I20260619 20:31:50.200096 6158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.66@KRBTEST.COM'} at 127.4.83.66:33469
I20260619 20:31:50.208069 6209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:50.200200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:46661 (local address 127.4.83.126:44167)
0619 20:31:50.200260 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:50.200262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:50.200493 (+ 231us) server_negotiation.cc:407] Connection header received
0619 20:31:50.200523 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:50.200523 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:50.200538 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:50.200621 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:50.201098 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.201622 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.202684 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.202992 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.203092 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:50.204583 (+ 1491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:50.204584 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:50.204585 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:50.204593 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:50.206234 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:50.206793 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:50.206795 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:50.206795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:50.206903 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:50.207283 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:50.207284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:50.207284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:50.207495 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:50.207571 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:50.207879 (+ 308us) server_negotiation.cc:299] Negotiation successful
0619 20:31:50.207988 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 20:31:50.208156 6508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:50.200131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:44167 (local address 127.4.83.66:46661)
0619 20:31:50.200361 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:50.200375 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:50.200447 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:50.200635 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:50.200636 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:50.200892 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:31:50.201005 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:50.201009 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.201741 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.201742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:50.202544 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:50.202548 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.203143 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.203146 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:50.203304 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:50.203922 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:31:50.203928 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:31:50.204449 (+ 521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:31:50.206360 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:50.206361 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:50.206362 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:50.206671 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:50.206991 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:50.206992 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:50.206993 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:50.207079 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:50.207583 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:31:50.207584 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:31:50.207683 (+ 99us) client_negotiation.cc:769] Sending connection context
0619 20:31:50.207769 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 20:31:50.208000 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":99,"threads_started":1}
WARNING: no policy specified for kudu/127.4.83.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
I20260619 20:31:50.232599 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:0
--local_ip_for_outbound_sockets=127.4.83.67
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:44167
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:45911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:31:50.323549 6511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:31:50.323858 6511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:31:50.323904 6511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:31:50.325492 6511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:31:50.325568 6511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:31:50.325603 6511 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 20:31:50.325620 6511 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 20:31:50.325651 6511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:31:50.327415 6511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:45911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.83.126:44167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:31:50.327706 6511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:31:50.328016 6511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:31:50.333864 6517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:50.333956 6516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:50.334371 6511 server_base.cc:1034] running on GCE node
W20260619 20:31:50.333956 6519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:50.334828 6511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:31:50.335103 6511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:31:50.336295 6511 hybrid_clock.cc:648] HybridClock initialized: now 1781901110336255 us; error 66 us; skew 500 ppm
Jun 19 20:31:50 dist-test-slave-4m58 krb5kdc[6114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901110, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:31:50.339461 6511 init.cc:377] Logged in from keytab as kudu/127.4.83.67@KRBTEST.COM (short username kudu)
I20260619 20:31:50.340065 6511 webserver.cc:460] Webserver started at http://127.4.83.67:38035/ using document root <none> and password file <none>
I20260619 20:31:50.340296 6511 fs_manager.cc:359] Metadata directory not provided
I20260619 20:31:50.340350 6511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:31:50.340484 6511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:31:50.341569 6511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-2/data/instance:
uuid: "5bbd2f6ea420477192319afe8301bcf4"
format_stamp: "Formatted at 2026-06-19 20:31:50 on dist-test-slave-4m58"
server_key: "3d243b6b1c1fbca38c096b747e8edcb6"
server_key_iv: "c9305eded1765eed094aa3da7a5715e9"
server_key_version: "encryptionkey@0"
I20260619 20:31:50.341971 6511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance:
uuid: "5bbd2f6ea420477192319afe8301bcf4"
format_stamp: "Formatted at 2026-06-19 20:31:50 on dist-test-slave-4m58"
server_key: "3d243b6b1c1fbca38c096b747e8edcb6"
server_key_iv: "c9305eded1765eed094aa3da7a5715e9"
server_key_version: "encryptionkey@0"
I20260619 20:31:50.343564 6511 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:31:50.344622 6526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:50.344941 6511 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:31:50.345062 6511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "5bbd2f6ea420477192319afe8301bcf4"
format_stamp: "Formatted at 2026-06-19 20:31:50 on dist-test-slave-4m58"
server_key: "3d243b6b1c1fbca38c096b747e8edcb6"
server_key_iv: "c9305eded1765eed094aa3da7a5715e9"
server_key_version: "encryptionkey@0"
I20260619 20:31:50.345142 6511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:31:50.386204 6511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:31:50.389508 6511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:31:50.389667 6511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:31:50.390287 6511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:31:50.390331 6511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:50.390362 6511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:31:50.390381 6511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:50.397763 6511 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:37647
I20260619 20:31:50.397837 6641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:37647 every 8 connection(s)
I20260619 20:31:50.398169 6511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
Jun 19 20:31:50 dist-test-slave-4m58 krb5kdc[6114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901110, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:31:50.399405 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 6511
I20260619 20:31:50.399549 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance
I20260619 20:31:50.399787 4429 external_mini_cluster.cc:1428] Setting key 170e114136359689a623415e54a4f69c
I20260619 20:31:50.402516 6209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:50.391215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:40543 (local address 127.4.83.126:44167)
0619 20:31:50.391292 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:50.391294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:50.391523 (+ 229us) server_negotiation.cc:407] Connection header received
0619 20:31:50.392464 (+ 941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:50.392466 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:50.392483 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:50.392584 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:50.393537 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.394228 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.395409 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.395659 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.395726 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:50.398886 (+ 3160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:50.398888 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:50.398889 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:50.398898 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:50.400843 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:50.401418 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:50.401419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:50.401420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:50.401490 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:50.401829 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:50.401829 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:50.401830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:50.401963 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:50.402030 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:50.402372 (+ 342us) server_negotiation.cc:299] Negotiation successful
0619 20:31:50.402448 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 20:31:50.402827 6535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:50.391162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:44167 (local address 127.4.83.67:40543)
0619 20:31:50.391367 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:50.391414 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:50.392300 (+ 886us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:50.392691 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:50.392696 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:50.393136 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:31:50.393406 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:50.393413 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.394325 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.394326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:50.395298 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:50.395302 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.395748 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.395749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:50.395939 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:50.396671 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:31:50.396684 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:31:50.398723 (+ 2039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:31:50.400947 (+ 2224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:50.400962 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:50.400965 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:50.401314 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:50.401601 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:50.401601 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:50.401602 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:50.401757 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:50.402060 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:31:50.402069 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:31:50.402242 (+ 173us) client_negotiation.cc:769] Sending connection context
0619 20:31:50.402346 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 20:31:50.402521 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":144,"threads_started":1}
I20260619 20:31:50.403419 6642 heartbeater.cc:347] Connected to a master server at 127.4.83.126:44167
I20260619 20:31:50.403517 6642 heartbeater.cc:464] Registering TS with master...
I20260619 20:31:50.403764 6642 heartbeater.cc:511] Master 127.4.83.126:44167 requested a full tablet report, sending...
I20260619 20:31:50.404299 6159 ts_manager.cc:194] Registered new tserver with Master: 5bbd2f6ea420477192319afe8301bcf4 (127.4.83.67:37647)
I20260619 20:31:50.404825 6159 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.67@KRBTEST.COM'} at 127.4.83.67:40543
I20260619 20:31:50.411549 6209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:50.404806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:33069 (local address 127.4.83.126:44167)
0619 20:31:50.404879 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:50.404881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:50.405036 (+ 155us) server_negotiation.cc:407] Connection header received
0619 20:31:50.405109 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:50.405110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:50.405119 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:50.405171 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:50.405651 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.406212 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.407138 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.407355 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.407436 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:50.408710 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:50.408711 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:50.408712 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:50.408720 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:50.410061 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:50.410471 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:50.410471 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:50.410471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:50.410525 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:50.410846 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:50.410847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:50.410847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:50.410984 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:50.411058 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:50.411339 (+ 281us) server_negotiation.cc:299] Negotiation successful
0619 20:31:50.411475 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 20:31:50.411586 6648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:50.404727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:44167 (local address 127.4.83.67:33069)
0619 20:31:50.404927 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:50.404942 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:50.405048 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:50.405180 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:50.405181 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:50.405437 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:31:50.405538 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:50.405540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.406353 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.406354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:50.406953 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:50.406956 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.407476 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.407477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:50.407560 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:50.408168 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:31:50.408173 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:31:50.408562 (+ 389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:31:50.410174 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:50.410176 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:50.410176 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:50.410398 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:50.410623 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:50.410625 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:50.410625 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:50.410721 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:50.411104 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:31:50.411105 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:31:50.411169 (+ 64us) client_negotiation.cc:769] Sending connection context
0619 20:31:50.411276 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 20:31:50.411446 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":123,"threads_started":1}
I20260619 20:31:50.412380 4429 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 20:31:50 dist-test-slave-4m58 krb5kdc[6114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 20:31:50 dist-test-slave-4m58 krb5kdc[6114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:31:50.440785 6209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:50.432081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53488 (local address 127.4.83.126:44167)
0619 20:31:50.432190 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:50.432192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:50.432206 (+ 14us) server_negotiation.cc:407] Connection header received
0619 20:31:50.432301 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:50.432302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:50.432313 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:50.432387 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:50.433138 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.433657 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.434725 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.434969 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.435046 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:50.437879 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:50.437880 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:50.437881 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:50.437889 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:50.439225 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:50.439834 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:50.439856 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:50.439856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:50.439920 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:50.440142 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:50.440143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:50.440143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:50.440272 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:50.440365 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:50.440604 (+ 239us) server_negotiation.cc:299] Negotiation successful
0619 20:31:50.440693 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 20:31:50.443177 6159 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53488:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:31:50.445132 6159 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:31:50.453095 6659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:50.448462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:40339 (local address 127.0.0.1:50352)
0619 20:31:50.448786 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:50.448822 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:50.448903 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:50.449355 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:50.449356 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:50.449359 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:31:50.449464 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:50.449467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.450746 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.450748 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:50.451860 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:50.451864 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.452715 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.452716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:50.452805 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:50.452821 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 20:31:50.452877 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 20:31:50.452976 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":4,"thread_start_us":93,"threads_started":1}
I20260619 20:31:50.453205 6662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:50.448590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46734 (local address 127.4.83.65:45613)
0619 20:31:50.449081 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:50.449087 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:50.449111 (+ 24us) server_negotiation.cc:407] Connection header received
0619 20:31:50.449157 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:50.449159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:50.449178 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:50.449253 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:31:50.449468 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.450227 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.451775 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.452573 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.452662 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:50.452741 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:50.453007 (+ 266us) server_negotiation.cc:299] Negotiation successful
0619 20:31:50.453076 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":105,"threads_started":1}
I20260619 20:31:50.453243 6663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:50.448750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51444 (local address 127.4.83.67:37647)
0619 20:31:50.449160 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:50.449166 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:50.449177 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:31:50.449220 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:50.449222 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:50.449273 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:50.449349 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:31:50.449457 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.450613 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.452141 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.452841 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.452904 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:50.452952 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:50.453084 (+ 132us) server_negotiation.cc:299] Negotiation successful
0619 20:31:50.453121 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":96,"threads_started":1}
I20260619 20:31:50.453248 6661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:50.448462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:37647 (local address 127.0.0.1:51444)
0619 20:31:50.448715 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:50.448992 (+ 277us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:50.449039 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:50.449354 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:50.449355 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:50.449358 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:31:50.449395 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:50.449396 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.450715 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.450716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:50.452045 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:50.452048 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.452926 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.452928 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:50.453009 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:50.453015 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 20:31:50.453065 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 20:31:50.453131 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":103,"thread_start_us":104,"threads_started":1}
I20260619 20:31:50.453843 6664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:50.448793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50352 (local address 127.4.83.66:40339)
0619 20:31:50.449164 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:50.449167 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:50.449178 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:31:50.449220 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:50.449222 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:50.449238 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:50.449540 (+ 302us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:31:50.449591 (+ 51us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.450643 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.451967 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.452600 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.453529 (+ 929us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:50.453645 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:50.453673 (+ 28us) server_negotiation.cc:299] Negotiation successful
0619 20:31:50.453730 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":79,"threads_started":1}
I20260619 20:31:50.454036 6660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:50.448472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:45613 (local address 127.0.0.1:46734)
0619 20:31:50.448988 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:50.449003 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:50.449067 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:50.449279 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:50.449282 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:50.449285 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:31:50.449384 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:50.449386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.450321 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.450322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:50.451649 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:50.451652 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.452682 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.452684 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:50.452800 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:50.452821 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 20:31:50.452899 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 20:31:50.453959 (+ 1060us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":998,"thread_start_us":150,"threads_started":1}
I20260619 20:31:50.454383 6576 tablet_service.cc:1467] Processing CreateTablet for tablet a0c93605976349d18aa9a31c3e91a48e (DEFAULT_TABLE table=test-table [id=c9c32ba644ed426bb4f700c20d439597]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:31:50.454433 6436 tablet_service.cc:1467] Processing CreateTablet for tablet a0c93605976349d18aa9a31c3e91a48e (DEFAULT_TABLE table=test-table [id=c9c32ba644ed426bb4f700c20d439597]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:31:50.454797 6436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0c93605976349d18aa9a31c3e91a48e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:50.454797 6576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0c93605976349d18aa9a31c3e91a48e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:50.454852 6296 tablet_service.cc:1467] Processing CreateTablet for tablet a0c93605976349d18aa9a31c3e91a48e (DEFAULT_TABLE table=test-table [id=c9c32ba644ed426bb4f700c20d439597]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:31:50.455325 6296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0c93605976349d18aa9a31c3e91a48e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:50.458606 6666 tablet_bootstrap.cc:492] T a0c93605976349d18aa9a31c3e91a48e P 7482052ce1824e138d8b6e9b5b48e24d: Bootstrap starting.
I20260619 20:31:50.458901 6665 tablet_bootstrap.cc:492] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4: Bootstrap starting.
I20260619 20:31:50.458938 6667 tablet_bootstrap.cc:492] T a0c93605976349d18aa9a31c3e91a48e P e372f443fb0b481d9ca913ba70fb63c2: Bootstrap starting.
I20260619 20:31:50.459336 6666 tablet_bootstrap.cc:654] T a0c93605976349d18aa9a31c3e91a48e P 7482052ce1824e138d8b6e9b5b48e24d: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:50.459703 6666 log.cc:824] T a0c93605976349d18aa9a31c3e91a48e P 7482052ce1824e138d8b6e9b5b48e24d: Log is configured to *not* fsync() on all Append() calls
I20260619 20:31:50.459805 6665 tablet_bootstrap.cc:654] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:50.460004 6667 tablet_bootstrap.cc:654] T a0c93605976349d18aa9a31c3e91a48e P e372f443fb0b481d9ca913ba70fb63c2: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:50.460146 6665 log.cc:824] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4: Log is configured to *not* fsync() on all Append() calls
I20260619 20:31:50.460395 6667 log.cc:824] T a0c93605976349d18aa9a31c3e91a48e P e372f443fb0b481d9ca913ba70fb63c2: Log is configured to *not* fsync() on all Append() calls
I20260619 20:31:50.460563 6666 tablet_bootstrap.cc:492] T a0c93605976349d18aa9a31c3e91a48e P 7482052ce1824e138d8b6e9b5b48e24d: No bootstrap required, opened a new log
I20260619 20:31:50.460630 6666 ts_tablet_manager.cc:1397] T a0c93605976349d18aa9a31c3e91a48e P 7482052ce1824e138d8b6e9b5b48e24d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:31:50.461026 6665 tablet_bootstrap.cc:492] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4: No bootstrap required, opened a new log
I20260619 20:31:50.461083 6665 ts_tablet_manager.cc:1397] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:31:50.462996 6666 raft_consensus.cc:348] T a0c93605976349d18aa9a31c3e91a48e P 7482052ce1824e138d8b6e9b5b48e24d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } } peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } }
I20260619 20:31:50.463213 6666 raft_consensus.cc:374] T a0c93605976349d18aa9a31c3e91a48e P 7482052ce1824e138d8b6e9b5b48e24d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:50.463266 6666 raft_consensus.cc:729] T a0c93605976349d18aa9a31c3e91a48e P 7482052ce1824e138d8b6e9b5b48e24d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7482052ce1824e138d8b6e9b5b48e24d, State: Initialized, Role: FOLLOWER
I20260619 20:31:50.463188 6665 raft_consensus.cc:348] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } } peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } }
I20260619 20:31:50.463330 6665 raft_consensus.cc:374] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:50.463359 6665 raft_consensus.cc:729] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bbd2f6ea420477192319afe8301bcf4, State: Initialized, Role: FOLLOWER
I20260619 20:31:50.463451 6665 consensus_queue.cc:260] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } } peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } }
I20260619 20:31:50.463407 6666 consensus_queue.cc:260] T a0c93605976349d18aa9a31c3e91a48e P 7482052ce1824e138d8b6e9b5b48e24d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } } peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } }
I20260619 20:31:50.463750 6665 ts_tablet_manager.cc:1428] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:31:50.463750 6666 ts_tablet_manager.cc:1428] T a0c93605976349d18aa9a31c3e91a48e P 7482052ce1824e138d8b6e9b5b48e24d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:31:50.463936 6502 heartbeater.cc:503] Master 127.4.83.126:44167 was elected leader, sending a full tablet report...
I20260619 20:31:50.463932 6642 heartbeater.cc:503] Master 127.4.83.126:44167 was elected leader, sending a full tablet report...
I20260619 20:31:50.465148 6667 tablet_bootstrap.cc:492] T a0c93605976349d18aa9a31c3e91a48e P e372f443fb0b481d9ca913ba70fb63c2: No bootstrap required, opened a new log
I20260619 20:31:50.465237 6667 ts_tablet_manager.cc:1397] T a0c93605976349d18aa9a31c3e91a48e P e372f443fb0b481d9ca913ba70fb63c2: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260619 20:31:50.466799 6667 raft_consensus.cc:348] T a0c93605976349d18aa9a31c3e91a48e P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } } peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } }
I20260619 20:31:50.466962 6667 raft_consensus.cc:374] T a0c93605976349d18aa9a31c3e91a48e P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:50.466992 6667 raft_consensus.cc:729] T a0c93605976349d18aa9a31c3e91a48e P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e372f443fb0b481d9ca913ba70fb63c2, State: Initialized, Role: FOLLOWER
I20260619 20:31:50.467113 6667 consensus_queue.cc:260] T a0c93605976349d18aa9a31c3e91a48e P e372f443fb0b481d9ca913ba70fb63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } } peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } }
I20260619 20:31:50.467392 6667 ts_tablet_manager.cc:1428] T a0c93605976349d18aa9a31c3e91a48e P e372f443fb0b481d9ca913ba70fb63c2: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 20:31:50.467523 6362 heartbeater.cc:503] Master 127.4.83.126:44167 was elected leader, sending a full tablet report...
W20260619 20:31:50.488107 6363 tablet.cc:2369] T a0c93605976349d18aa9a31c3e91a48e P e372f443fb0b481d9ca913ba70fb63c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:31:50.556577 6671 raft_consensus.cc:482] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:31:50.556758 6671 raft_consensus.cc:504] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } } peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } }
I20260619 20:31:50.557116 6671 leader_election.cc:288] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e372f443fb0b481d9ca913ba70fb63c2 (127.4.83.65:45613), 7482052ce1824e138d8b6e9b5b48e24d (127.4.83.66:40339)
I20260619 20:31:50.560485 6535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:50.557418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:45613 (local address 127.4.83.67:39713)
0619 20:31:50.557504 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:50.557518 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:50.557588 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:50.557853 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:50.557854 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:50.557856 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:31:50.557956 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:50.557959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.558748 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.558749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:50.559620 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:50.559622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.560207 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.560208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:50.560294 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:50.560299 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:31:50.560355 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 20:31:50.560401 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":3}
I20260619 20:31:50.560525 6662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:50.557471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:39713 (local address 127.4.83.65:45613)
0619 20:31:50.557531 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:50.557532 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:50.557572 (+ 40us) server_negotiation.cc:407] Connection header received
0619 20:31:50.557664 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:50.557665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:50.557679 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:50.557840 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:31:50.558036 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.558656 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.559738 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.560127 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.560186 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:50.560196 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:50.560391 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 20:31:50.560443 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 20:31:50.560695 6664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:50.557468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:47127 (local address 127.4.83.66:40339)
0619 20:31:50.557518 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:50.557520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:50.557811 (+ 291us) server_negotiation.cc:407] Connection header received
0619 20:31:50.557878 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:50.557879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:50.557894 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:50.557955 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:31:50.558148 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.558646 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.559748 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.560227 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.560306 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:50.560319 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:50.560573 (+ 254us) server_negotiation.cc:299] Negotiation successful
0619 20:31:50.560621 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 20:31:50.560794 6674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:50.557418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:40339 (local address 127.4.83.67:47127)
0619 20:31:50.557714 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:50.557728 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:50.557807 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:50.557975 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:50.557976 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:50.557978 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:31:50.558073 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:50.558075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.558729 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.558731 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:50.559641 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:50.559644 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.560366 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.560367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:50.560469 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:50.560476 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 20:31:50.560559 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 20:31:50.560671 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":105,"thread_start_us":140,"threads_started":1}
I20260619 20:31:50.561129 6316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0c93605976349d18aa9a31c3e91a48e" candidate_uuid: "5bbd2f6ea420477192319afe8301bcf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e372f443fb0b481d9ca913ba70fb63c2" is_pre_election: true
I20260619 20:31:50.561249 6456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0c93605976349d18aa9a31c3e91a48e" candidate_uuid: "5bbd2f6ea420477192319afe8301bcf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7482052ce1824e138d8b6e9b5b48e24d" is_pre_election: true
I20260619 20:31:50.561391 6456 raft_consensus.cc:2454] T a0c93605976349d18aa9a31c3e91a48e P 7482052ce1824e138d8b6e9b5b48e24d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bbd2f6ea420477192319afe8301bcf4 in term 0.
I20260619 20:31:50.561388 6316 raft_consensus.cc:2454] T a0c93605976349d18aa9a31c3e91a48e P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bbd2f6ea420477192319afe8301bcf4 in term 0.
I20260619 20:31:50.561666 6527 leader_election.cc:302] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5bbd2f6ea420477192319afe8301bcf4, 7482052ce1824e138d8b6e9b5b48e24d; no voters:
I20260619 20:31:50.561904 6671 raft_consensus.cc:2781] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:31:50.561951 6671 raft_consensus.cc:482] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:31:50.561967 6671 raft_consensus.cc:3037] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:50.562913 6671 raft_consensus.cc:504] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } } peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } }
I20260619 20:31:50.563086 6671 leader_election.cc:288] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4 [CANDIDATE]: Term 1 election: Requested vote from peers e372f443fb0b481d9ca913ba70fb63c2 (127.4.83.65:45613), 7482052ce1824e138d8b6e9b5b48e24d (127.4.83.66:40339)
I20260619 20:31:50.563341 6316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0c93605976349d18aa9a31c3e91a48e" candidate_uuid: "5bbd2f6ea420477192319afe8301bcf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e372f443fb0b481d9ca913ba70fb63c2"
I20260619 20:31:50.563459 6456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0c93605976349d18aa9a31c3e91a48e" candidate_uuid: "5bbd2f6ea420477192319afe8301bcf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7482052ce1824e138d8b6e9b5b48e24d"
I20260619 20:31:50.563491 6316 raft_consensus.cc:3037] T a0c93605976349d18aa9a31c3e91a48e P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:50.563608 6456 raft_consensus.cc:3037] T a0c93605976349d18aa9a31c3e91a48e P 7482052ce1824e138d8b6e9b5b48e24d [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:50.564610 6456 raft_consensus.cc:2454] T a0c93605976349d18aa9a31c3e91a48e P 7482052ce1824e138d8b6e9b5b48e24d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bbd2f6ea420477192319afe8301bcf4 in term 1.
I20260619 20:31:50.564707 6316 raft_consensus.cc:2454] T a0c93605976349d18aa9a31c3e91a48e P e372f443fb0b481d9ca913ba70fb63c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bbd2f6ea420477192319afe8301bcf4 in term 1.
I20260619 20:31:50.564883 6527 leader_election.cc:302] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5bbd2f6ea420477192319afe8301bcf4, 7482052ce1824e138d8b6e9b5b48e24d; no voters:
I20260619 20:31:50.565092 6671 raft_consensus.cc:2781] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:31:50.565315 6671 raft_consensus.cc:686] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4 [term 1 LEADER]: Becoming Leader. State: Replica: 5bbd2f6ea420477192319afe8301bcf4, State: Running, Role: LEADER
I20260619 20:31:50.565441 6671 consensus_queue.cc:237] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } } peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } }
I20260619 20:31:50.566471 6159 catalog_manager.cc:5495] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5bbd2f6ea420477192319afe8301bcf4 (127.4.83.67). New cstate: current_term: 1 leader_uuid: "5bbd2f6ea420477192319afe8301bcf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } health_report { overall_health: UNKNOWN } } }
Jun 19 20:31:50 dist-test-slave-4m58 krb5kdc[6114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901109, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:31:50.626528 6682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:50.616873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:44167 (local address 127.0.0.1:53500)
0619 20:31:50.617139 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:50.617153 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:50.617259 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:50.617444 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:50.617445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:50.617965 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:31:50.618043 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:50.618047 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.618783 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.618784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:50.619487 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:50.619491 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.619983 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.619985 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:50.620075 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:50.620880 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:31:50.620886 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:31:50.622964 (+ 2078us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:31:50.624851 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:50.624872 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:50.624876 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:50.625256 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:50.625611 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:50.625612 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:50.625612 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:50.625697 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:50.626041 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:31:50.626062 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:31:50.626140 (+ 78us) client_negotiation.cc:769] Sending connection context
0619 20:31:50.626251 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 20:31:50.626411 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":131,"threads_started":1}
I20260619 20:31:50.626608 6209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:50.616967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53500 (local address 127.4.83.126:44167)
0619 20:31:50.617056 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:50.617060 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:50.617314 (+ 254us) server_negotiation.cc:407] Connection header received
0619 20:31:50.617351 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:50.617352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:50.617366 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:50.617419 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:50.618162 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.618663 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.619603 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.619886 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.619962 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:50.623135 (+ 3173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:50.623138 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:50.623138 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:50.623149 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:50.624717 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:50.625406 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:50.625408 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:50.625409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:50.625501 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:50.625799 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:50.625800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:50.625800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:50.625946 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:50.626020 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:50.626320 (+ 300us) server_negotiation.cc:299] Negotiation successful
0619 20:31:50.626484 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":56}
I20260619 20:31:50.629313 6159 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:53500:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 20:31:50.629546 6159 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:31:50.631744 6296 tablet_service.cc:1467] Processing CreateTablet for tablet 3dee8b52db3a4bbea1bdeaef459c095b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=389cfcd20ec8407385e39c5ffc39cb15]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 20:31:50.631744 6436 tablet_service.cc:1467] Processing CreateTablet for tablet 3dee8b52db3a4bbea1bdeaef459c095b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=389cfcd20ec8407385e39c5ffc39cb15]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 20:31:50.631995 6296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dee8b52db3a4bbea1bdeaef459c095b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:50.631995 6436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dee8b52db3a4bbea1bdeaef459c095b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:50.632090 6576 tablet_service.cc:1467] Processing CreateTablet for tablet 3dee8b52db3a4bbea1bdeaef459c095b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=389cfcd20ec8407385e39c5ffc39cb15]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 20:31:50.632345 6576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dee8b52db3a4bbea1bdeaef459c095b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:50.633831 6667 tablet_bootstrap.cc:492] T 3dee8b52db3a4bbea1bdeaef459c095b P e372f443fb0b481d9ca913ba70fb63c2: Bootstrap starting.
I20260619 20:31:50.633853 6665 tablet_bootstrap.cc:492] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4: Bootstrap starting.
I20260619 20:31:50.633841 6666 tablet_bootstrap.cc:492] T 3dee8b52db3a4bbea1bdeaef459c095b P 7482052ce1824e138d8b6e9b5b48e24d: Bootstrap starting.
I20260619 20:31:50.634308 6667 tablet_bootstrap.cc:654] T 3dee8b52db3a4bbea1bdeaef459c095b P e372f443fb0b481d9ca913ba70fb63c2: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:50.634502 6665 tablet_bootstrap.cc:654] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:50.634717 6666 tablet_bootstrap.cc:654] T 3dee8b52db3a4bbea1bdeaef459c095b P 7482052ce1824e138d8b6e9b5b48e24d: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:50.635380 6665 tablet_bootstrap.cc:492] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4: No bootstrap required, opened a new log
I20260619 20:31:50.635448 6665 ts_tablet_manager.cc:1397] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:31:50.635572 6666 tablet_bootstrap.cc:492] T 3dee8b52db3a4bbea1bdeaef459c095b P 7482052ce1824e138d8b6e9b5b48e24d: No bootstrap required, opened a new log
I20260619 20:31:50.635620 6667 tablet_bootstrap.cc:492] T 3dee8b52db3a4bbea1bdeaef459c095b P e372f443fb0b481d9ca913ba70fb63c2: No bootstrap required, opened a new log
I20260619 20:31:50.635639 6666 ts_tablet_manager.cc:1397] T 3dee8b52db3a4bbea1bdeaef459c095b P 7482052ce1824e138d8b6e9b5b48e24d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:31:50.635612 6665 raft_consensus.cc:348] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:50.635675 6667 ts_tablet_manager.cc:1397] T 3dee8b52db3a4bbea1bdeaef459c095b P e372f443fb0b481d9ca913ba70fb63c2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 20:31:50.635717 6665 raft_consensus.cc:374] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:50.635739 6665 raft_consensus.cc:729] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bbd2f6ea420477192319afe8301bcf4, State: Initialized, Role: FOLLOWER
I20260619 20:31:50.635797 6665 consensus_queue.cc:260] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:50.635921 6665 ts_tablet_manager.cc:1428] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 20:31:50.635869 6666 raft_consensus.cc:348] T 3dee8b52db3a4bbea1bdeaef459c095b P 7482052ce1824e138d8b6e9b5b48e24d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:50.635906 6667 raft_consensus.cc:348] T 3dee8b52db3a4bbea1bdeaef459c095b P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:50.635963 6666 raft_consensus.cc:374] T 3dee8b52db3a4bbea1bdeaef459c095b P 7482052ce1824e138d8b6e9b5b48e24d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:50.635979 6667 raft_consensus.cc:374] T 3dee8b52db3a4bbea1bdeaef459c095b P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:50.635993 6666 raft_consensus.cc:729] T 3dee8b52db3a4bbea1bdeaef459c095b P 7482052ce1824e138d8b6e9b5b48e24d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7482052ce1824e138d8b6e9b5b48e24d, State: Initialized, Role: FOLLOWER
I20260619 20:31:50.636003 6667 raft_consensus.cc:729] T 3dee8b52db3a4bbea1bdeaef459c095b P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e372f443fb0b481d9ca913ba70fb63c2, State: Initialized, Role: FOLLOWER
I20260619 20:31:50.636049 6666 consensus_queue.cc:260] T 3dee8b52db3a4bbea1bdeaef459c095b P 7482052ce1824e138d8b6e9b5b48e24d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:50.636057 6667 consensus_queue.cc:260] T 3dee8b52db3a4bbea1bdeaef459c095b P e372f443fb0b481d9ca913ba70fb63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:50.636092 6671 tablet_replica.cc:439] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } } }
I20260619 20:31:50.636148 6666 ts_tablet_manager.cc:1428] T 3dee8b52db3a4bbea1bdeaef459c095b P 7482052ce1824e138d8b6e9b5b48e24d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 20:31:50.636180 6667 ts_tablet_manager.cc:1428] T 3dee8b52db3a4bbea1bdeaef459c095b P e372f443fb0b481d9ca913ba70fb63c2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:50.636171 6671 tablet_replica.cc:442] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:31:50.636327 6672 tablet_replica.cc:439] T 3dee8b52db3a4bbea1bdeaef459c095b P 7482052ce1824e138d8b6e9b5b48e24d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } } }
I20260619 20:31:50.636396 6672 tablet_replica.cc:442] T 3dee8b52db3a4bbea1bdeaef459c095b P 7482052ce1824e138d8b6e9b5b48e24d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:31:50.636577 6673 tablet_replica.cc:439] T 3dee8b52db3a4bbea1bdeaef459c095b P e372f443fb0b481d9ca913ba70fb63c2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } } }
I20260619 20:31:50.636678 6673 tablet_replica.cc:442] T 3dee8b52db3a4bbea1bdeaef459c095b P e372f443fb0b481d9ca913ba70fb63c2: This TxnStatusTablet replica's current role is: FOLLOWER
W20260619 20:31:50.648725 6643 tablet.cc:2369] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:31:50.693706 6503 tablet.cc:2369] T 3dee8b52db3a4bbea1bdeaef459c095b P 7482052ce1824e138d8b6e9b5b48e24d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:31:50.752816 6671 raft_consensus.cc:482] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:31:50.752959 6671 raft_consensus.cc:504] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:50.753193 6671 leader_election.cc:288] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e372f443fb0b481d9ca913ba70fb63c2 (127.4.83.65:45613), 7482052ce1824e138d8b6e9b5b48e24d (127.4.83.66:40339)
I20260619 20:31:50.753507 6456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dee8b52db3a4bbea1bdeaef459c095b" candidate_uuid: "5bbd2f6ea420477192319afe8301bcf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7482052ce1824e138d8b6e9b5b48e24d" is_pre_election: true
I20260619 20:31:50.753508 6316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dee8b52db3a4bbea1bdeaef459c095b" candidate_uuid: "5bbd2f6ea420477192319afe8301bcf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e372f443fb0b481d9ca913ba70fb63c2" is_pre_election: true
I20260619 20:31:50.753648 6456 raft_consensus.cc:2454] T 3dee8b52db3a4bbea1bdeaef459c095b P 7482052ce1824e138d8b6e9b5b48e24d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bbd2f6ea420477192319afe8301bcf4 in term 0.
I20260619 20:31:50.753648 6316 raft_consensus.cc:2454] T 3dee8b52db3a4bbea1bdeaef459c095b P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bbd2f6ea420477192319afe8301bcf4 in term 0.
I20260619 20:31:50.753860 6527 leader_election.cc:302] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5bbd2f6ea420477192319afe8301bcf4, 7482052ce1824e138d8b6e9b5b48e24d; no voters:
I20260619 20:31:50.753969 6671 raft_consensus.cc:2781] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:31:50.754017 6671 raft_consensus.cc:482] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:31:50.754034 6671 raft_consensus.cc:3037] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:50.754807 6671 raft_consensus.cc:504] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:50.754952 6671 leader_election.cc:288] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4 [CANDIDATE]: Term 1 election: Requested vote from peers e372f443fb0b481d9ca913ba70fb63c2 (127.4.83.65:45613), 7482052ce1824e138d8b6e9b5b48e24d (127.4.83.66:40339)
I20260619 20:31:50.755168 6316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dee8b52db3a4bbea1bdeaef459c095b" candidate_uuid: "5bbd2f6ea420477192319afe8301bcf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e372f443fb0b481d9ca913ba70fb63c2"
I20260619 20:31:50.755187 6456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dee8b52db3a4bbea1bdeaef459c095b" candidate_uuid: "5bbd2f6ea420477192319afe8301bcf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7482052ce1824e138d8b6e9b5b48e24d"
I20260619 20:31:50.755303 6316 raft_consensus.cc:3037] T 3dee8b52db3a4bbea1bdeaef459c095b P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:50.755303 6456 raft_consensus.cc:3037] T 3dee8b52db3a4bbea1bdeaef459c095b P 7482052ce1824e138d8b6e9b5b48e24d [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:50.756323 6316 raft_consensus.cc:2454] T 3dee8b52db3a4bbea1bdeaef459c095b P e372f443fb0b481d9ca913ba70fb63c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bbd2f6ea420477192319afe8301bcf4 in term 1.
I20260619 20:31:50.756323 6456 raft_consensus.cc:2454] T 3dee8b52db3a4bbea1bdeaef459c095b P 7482052ce1824e138d8b6e9b5b48e24d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bbd2f6ea420477192319afe8301bcf4 in term 1.
I20260619 20:31:50.756557 6530 leader_election.cc:302] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5bbd2f6ea420477192319afe8301bcf4, e372f443fb0b481d9ca913ba70fb63c2; no voters:
I20260619 20:31:50.756712 6671 raft_consensus.cc:2781] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:31:50.756793 6671 raft_consensus.cc:686] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4 [term 1 LEADER]: Becoming Leader. State: Replica: 5bbd2f6ea420477192319afe8301bcf4, State: Running, Role: LEADER
I20260619 20:31:50.756851 6671 consensus_queue.cc:237] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:50.757156 6675 tablet_replica.cc:439] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4: TxnStatusTablet state changed. Reason: New leader 5bbd2f6ea420477192319afe8301bcf4. Latest consensus state: current_term: 1 leader_uuid: "5bbd2f6ea420477192319afe8301bcf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } } }
I20260619 20:31:50.757249 6675 tablet_replica.cc:442] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4: This TxnStatusTablet replica's current role is: LEADER
I20260619 20:31:50.757434 6687 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 20:31:50.757561 6687 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 20:31:50.757598 6159 catalog_manager.cc:5495] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5bbd2f6ea420477192319afe8301bcf4 (127.4.83.67). New cstate: current_term: 1 leader_uuid: "5bbd2f6ea420477192319afe8301bcf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } health_report { overall_health: HEALTHY } } }
I20260619 20:31:50.788928 6682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:50.786270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:37647 (local address 127.0.0.1:51456)
0619 20:31:50.786410 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:50.786425 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:50.786501 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:50.786723 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:50.786724 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:50.786725 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:31:50.786802 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:50.786805 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.787444 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.787445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:50.788087 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:50.788089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.788422 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.788423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:50.788494 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:50.788511 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:31:50.788756 (+ 245us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:31:50.788764 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 20:31:50.788803 (+ 39us) client_negotiation.cc:240] Negotiation successful
0619 20:31:50.788858 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260619 20:31:50.788920 6663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:50.786402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51456 (local address 127.4.83.67:37647)
0619 20:31:50.786480 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:50.786482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:50.786492 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:31:50.786599 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:50.786600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:50.786649 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:50.786708 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:31:50.786863 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.787366 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.788159 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.788361 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.788417 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:50.788595 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:31:50.788699 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:31:50.788769 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:50.788811 (+ 42us) server_negotiation.cc:299] Negotiation successful
0619 20:31:50.788834 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 20:31:50.792181 6316 raft_consensus.cc:1261] T 3dee8b52db3a4bbea1bdeaef459c095b P e372f443fb0b481d9ca913ba70fb63c2 [term 1 FOLLOWER]: Refusing update from remote peer 5bbd2f6ea420477192319afe8301bcf4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:31:50.792263 6456 raft_consensus.cc:1261] T 3dee8b52db3a4bbea1bdeaef459c095b P 7482052ce1824e138d8b6e9b5b48e24d [term 1 FOLLOWER]: Refusing update from remote peer 5bbd2f6ea420477192319afe8301bcf4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:31:50.792613 6675 consensus_queue.cc:1035] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:31:50.792801 6671 consensus_queue.cc:1035] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:31:50.794473 6672 tablet_replica.cc:439] T 3dee8b52db3a4bbea1bdeaef459c095b P 7482052ce1824e138d8b6e9b5b48e24d: TxnStatusTablet state changed. Reason: New leader 5bbd2f6ea420477192319afe8301bcf4. Latest consensus state: current_term: 1 leader_uuid: "5bbd2f6ea420477192319afe8301bcf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } } }
I20260619 20:31:50.794605 6672 tablet_replica.cc:442] T 3dee8b52db3a4bbea1bdeaef459c095b P 7482052ce1824e138d8b6e9b5b48e24d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:31:50.794881 6673 tablet_replica.cc:439] T 3dee8b52db3a4bbea1bdeaef459c095b P e372f443fb0b481d9ca913ba70fb63c2: TxnStatusTablet state changed. Reason: New leader 5bbd2f6ea420477192319afe8301bcf4. Latest consensus state: current_term: 1 leader_uuid: "5bbd2f6ea420477192319afe8301bcf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } } }
I20260619 20:31:50.794999 6673 tablet_replica.cc:442] T 3dee8b52db3a4bbea1bdeaef459c095b P e372f443fb0b481d9ca913ba70fb63c2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:31:50.795878 6671 tablet_replica.cc:439] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5bbd2f6ea420477192319afe8301bcf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } } }
I20260619 20:31:50.795984 6671 tablet_replica.cc:442] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4: This TxnStatusTablet replica's current role is: LEADER
I20260619 20:31:50.796026 6671 tablet_replica.cc:439] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5bbd2f6ea420477192319afe8301bcf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } } }
I20260619 20:31:50.796059 6671 tablet_replica.cc:442] T 3dee8b52db3a4bbea1bdeaef459c095b P 5bbd2f6ea420477192319afe8301bcf4: This TxnStatusTablet replica's current role is: LEADER
I20260619 20:31:50.796123 6690 mvcc.cc:205] Tried to move back new op lower bound from 7298666949800681472 to 7298666949660352512. Current Snapshot: MvccSnapshot[applied={T|T < 7298666949800681472}]
I20260619 20:31:50.796429 6673 tablet_replica.cc:439] T 3dee8b52db3a4bbea1bdeaef459c095b P e372f443fb0b481d9ca913ba70fb63c2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5bbd2f6ea420477192319afe8301bcf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } } }
I20260619 20:31:50.796510 6673 tablet_replica.cc:442] T 3dee8b52db3a4bbea1bdeaef459c095b P e372f443fb0b481d9ca913ba70fb63c2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:31:50.797009 6691 mvcc.cc:205] Tried to move back new op lower bound from 7298666949800681472 to 7298666949660352512. Current Snapshot: MvccSnapshot[applied={T|T < 7298666949800681472}]
I20260619 20:31:50.797376 6672 tablet_replica.cc:439] T 3dee8b52db3a4bbea1bdeaef459c095b P 7482052ce1824e138d8b6e9b5b48e24d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5bbd2f6ea420477192319afe8301bcf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } } }
I20260619 20:31:50.797499 6672 tablet_replica.cc:442] T 3dee8b52db3a4bbea1bdeaef459c095b P 7482052ce1824e138d8b6e9b5b48e24d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:31:50.797729 6692 mvcc.cc:205] Tried to move back new op lower bound from 7298666949800681472 to 7298666949660352512. Current Snapshot: MvccSnapshot[applied={T|T < 7298666949800681472}]
I20260619 20:31:50.801884 6663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:50.798814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51460 (local address 127.4.83.67:37647)
0619 20:31:50.798888 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:50.798891 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:50.798902 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:31:50.798929 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:50.798929 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:50.798940 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:50.798998 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:31:50.799209 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.799721 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.801008 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.801229 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.801313 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:50.801562 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:31:50.801610 (+ 48us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:31:50.801684 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:50.801781 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 20:31:50.801813 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 20:31:50.808494 6663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:50.805404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:57665 (local address 127.4.83.67:37647)
0619 20:31:50.805451 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:50.805452 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:50.805466 (+ 14us) server_negotiation.cc:407] Connection header received
0619 20:31:50.805536 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:50.805537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:50.805545 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:50.805603 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:31:50.805878 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.806371 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.807584 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.807784 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.807907 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:50.808162 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:31:50.808192 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:31:50.808234 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:50.808329 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 20:31:50.808352 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 20:31:50.808514 6648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:50.805340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:37647 (local address 127.4.83.67:57665)
0619 20:31:50.805417 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:50.805431 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:50.805481 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:50.805681 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:50.805682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:50.805684 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:31:50.805789 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:50.805791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.806451 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.806452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:50.807445 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:50.807448 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:50.807920 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:50.807921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:50.808000 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:50.808029 (+ 29us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:31:50.808251 (+ 222us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:31:50.808253 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:31:50.808302 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 20:31:50.808361 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260619 20:31:50.811414 6456 raft_consensus.cc:1261] T a0c93605976349d18aa9a31c3e91a48e P 7482052ce1824e138d8b6e9b5b48e24d [term 1 FOLLOWER]: Refusing update from remote peer 5bbd2f6ea420477192319afe8301bcf4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:31:50.811512 6316 raft_consensus.cc:1261] T a0c93605976349d18aa9a31c3e91a48e P e372f443fb0b481d9ca913ba70fb63c2 [term 1 FOLLOWER]: Refusing update from remote peer 5bbd2f6ea420477192319afe8301bcf4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:31:50.811623 6671 consensus_queue.cc:1035] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:31:50.811762 6671 consensus_queue.cc:1035] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:31:50.828199 6158 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53488:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:31:50.828325 6158 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53488:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:31:50.829159 6158 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1164ce5a83bc48bc91a00bac7c0d9552: Sending DeleteTablet for 3 replicas of tablet a0c93605976349d18aa9a31c3e91a48e
I20260619 20:31:50.829548 6436 tablet_service.cc:1514] Processing DeleteTablet for tablet a0c93605976349d18aa9a31c3e91a48e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:31:50 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:50352
I20260619 20:31:50.829602 6296 tablet_service.cc:1514] Processing DeleteTablet for tablet a0c93605976349d18aa9a31c3e91a48e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:31:50 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:46734
I20260619 20:31:50.829653 6575 tablet_service.cc:1514] Processing DeleteTablet for tablet a0c93605976349d18aa9a31c3e91a48e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:31:50 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:51444
I20260619 20:31:50.830039 6708 tablet_replica.cc:330] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4: stopping tablet replica
I20260619 20:31:50.830195 6708 raft_consensus.cc:2229] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:31:50.830346 6708 raft_consensus.cc:2258] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:31:50.830494 6158 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53488:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:31:50.830745 6158 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:31:50.831025 6708 ts_tablet_manager.cc:1905] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:31:50.832531 6707 tablet_replica.cc:330] T a0c93605976349d18aa9a31c3e91a48e P 7482052ce1824e138d8b6e9b5b48e24d: stopping tablet replica
I20260619 20:31:50.832742 6707 raft_consensus.cc:2229] T a0c93605976349d18aa9a31c3e91a48e P 7482052ce1824e138d8b6e9b5b48e24d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:31:50.832819 6707 raft_consensus.cc:2258] T a0c93605976349d18aa9a31c3e91a48e P 7482052ce1824e138d8b6e9b5b48e24d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:31:50.832885 6296 tablet_service.cc:1467] Processing CreateTablet for tablet 28b608d704d8479c9387a6a9732493f5 (DEFAULT_TABLE table=test-table [id=f63f664ef41b4055ae3defcc59ea3212]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:31:50.833024 6296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28b608d704d8479c9387a6a9732493f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:50.833317 6706 tablet_replica.cc:330] T a0c93605976349d18aa9a31c3e91a48e P e372f443fb0b481d9ca913ba70fb63c2: stopping tablet replica
I20260619 20:31:50.833451 6706 raft_consensus.cc:2229] T a0c93605976349d18aa9a31c3e91a48e P e372f443fb0b481d9ca913ba70fb63c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:31:50.833510 6706 raft_consensus.cc:2258] T a0c93605976349d18aa9a31c3e91a48e P e372f443fb0b481d9ca913ba70fb63c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:31:50.833525 6708 ts_tablet_manager.cc:1918] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:31:50.833604 6708 log.cc:1196] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-2/wal/wals/a0c93605976349d18aa9a31c3e91a48e
I20260619 20:31:50.833858 6708 ts_tablet_manager.cc:1939] T a0c93605976349d18aa9a31c3e91a48e P 5bbd2f6ea420477192319afe8301bcf4: Deleting consensus metadata
I20260619 20:31:50.834336 6667 tablet_bootstrap.cc:492] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2: Bootstrap starting.
I20260619 20:31:50.834460 6707 ts_tablet_manager.cc:1905] T a0c93605976349d18aa9a31c3e91a48e P 7482052ce1824e138d8b6e9b5b48e24d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:31:50.834578 6436 tablet_service.cc:1467] Processing CreateTablet for tablet 28b608d704d8479c9387a6a9732493f5 (DEFAULT_TABLE table=test-table [id=f63f664ef41b4055ae3defcc59ea3212]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:31:50.834784 6436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28b608d704d8479c9387a6a9732493f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:50.834797 6667 tablet_bootstrap.cc:654] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:50.835428 6667 tablet_bootstrap.cc:492] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2: No bootstrap required, opened a new log
I20260619 20:31:50.835469 6667 ts_tablet_manager.cc:1397] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:31:50.835615 6667 raft_consensus.cc:348] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:50.835690 6667 raft_consensus.cc:374] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:50.835712 6667 raft_consensus.cc:729] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e372f443fb0b481d9ca913ba70fb63c2, State: Initialized, Role: FOLLOWER
I20260619 20:31:50.835758 6667 consensus_queue.cc:260] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:50.835873 6667 ts_tablet_manager.cc:1428] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 20:31:50.836905 6666 tablet_bootstrap.cc:492] T 28b608d704d8479c9387a6a9732493f5 P 7482052ce1824e138d8b6e9b5b48e24d: Bootstrap starting.
I20260619 20:31:50.837373 6666 tablet_bootstrap.cc:654] T 28b608d704d8479c9387a6a9732493f5 P 7482052ce1824e138d8b6e9b5b48e24d: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:50.837675 6706 ts_tablet_manager.cc:1905] T a0c93605976349d18aa9a31c3e91a48e P e372f443fb0b481d9ca913ba70fb63c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:31:50.838034 6666 tablet_bootstrap.cc:492] T 28b608d704d8479c9387a6a9732493f5 P 7482052ce1824e138d8b6e9b5b48e24d: No bootstrap required, opened a new log
I20260619 20:31:50.838080 6666 ts_tablet_manager.cc:1397] T 28b608d704d8479c9387a6a9732493f5 P 7482052ce1824e138d8b6e9b5b48e24d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:31:50.838223 6666 raft_consensus.cc:348] T 28b608d704d8479c9387a6a9732493f5 P 7482052ce1824e138d8b6e9b5b48e24d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:50.838366 6666 raft_consensus.cc:374] T 28b608d704d8479c9387a6a9732493f5 P 7482052ce1824e138d8b6e9b5b48e24d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:50.838382 6666 raft_consensus.cc:729] T 28b608d704d8479c9387a6a9732493f5 P 7482052ce1824e138d8b6e9b5b48e24d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7482052ce1824e138d8b6e9b5b48e24d, State: Initialized, Role: FOLLOWER
I20260619 20:31:50.838426 6666 consensus_queue.cc:260] T 28b608d704d8479c9387a6a9732493f5 P 7482052ce1824e138d8b6e9b5b48e24d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:50.838518 6666 ts_tablet_manager.cc:1428] T 28b608d704d8479c9387a6a9732493f5 P 7482052ce1824e138d8b6e9b5b48e24d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 20:31:50.838793 6575 tablet_service.cc:1467] Processing CreateTablet for tablet 28b608d704d8479c9387a6a9732493f5 (DEFAULT_TABLE table=test-table [id=f63f664ef41b4055ae3defcc59ea3212]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:31:50.838984 6575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28b608d704d8479c9387a6a9732493f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:50.840402 6144 catalog_manager.cc:4841] TS 5bbd2f6ea420477192319afe8301bcf4 (127.4.83.67:37647): tablet a0c93605976349d18aa9a31c3e91a48e (table test-table [id=c9c32ba644ed426bb4f700c20d439597]) successfully deleted
I20260619 20:31:50.840731 6665 tablet_bootstrap.cc:492] T 28b608d704d8479c9387a6a9732493f5 P 5bbd2f6ea420477192319afe8301bcf4: Bootstrap starting.
I20260619 20:31:50.840823 6707 ts_tablet_manager.cc:1918] T a0c93605976349d18aa9a31c3e91a48e P 7482052ce1824e138d8b6e9b5b48e24d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:31:50.840925 6707 log.cc:1196] T a0c93605976349d18aa9a31c3e91a48e P 7482052ce1824e138d8b6e9b5b48e24d: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-1/wal/wals/a0c93605976349d18aa9a31c3e91a48e
I20260619 20:31:50.841176 6665 tablet_bootstrap.cc:654] T 28b608d704d8479c9387a6a9732493f5 P 5bbd2f6ea420477192319afe8301bcf4: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:50.841243 6707 ts_tablet_manager.cc:1939] T a0c93605976349d18aa9a31c3e91a48e P 7482052ce1824e138d8b6e9b5b48e24d: Deleting consensus metadata
I20260619 20:31:50.841751 6143 catalog_manager.cc:4841] TS 7482052ce1824e138d8b6e9b5b48e24d (127.4.83.66:40339): tablet a0c93605976349d18aa9a31c3e91a48e (table test-table [id=c9c32ba644ed426bb4f700c20d439597]) successfully deleted
I20260619 20:31:50.841825 6665 tablet_bootstrap.cc:492] T 28b608d704d8479c9387a6a9732493f5 P 5bbd2f6ea420477192319afe8301bcf4: No bootstrap required, opened a new log
I20260619 20:31:50.841867 6665 ts_tablet_manager.cc:1397] T 28b608d704d8479c9387a6a9732493f5 P 5bbd2f6ea420477192319afe8301bcf4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:31:50.842058 6665 raft_consensus.cc:348] T 28b608d704d8479c9387a6a9732493f5 P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:50.842144 6665 raft_consensus.cc:374] T 28b608d704d8479c9387a6a9732493f5 P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:50.842163 6665 raft_consensus.cc:729] T 28b608d704d8479c9387a6a9732493f5 P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bbd2f6ea420477192319afe8301bcf4, State: Initialized, Role: FOLLOWER
I20260619 20:31:50.842216 6665 consensus_queue.cc:260] T 28b608d704d8479c9387a6a9732493f5 P 5bbd2f6ea420477192319afe8301bcf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:50.842321 6665 ts_tablet_manager.cc:1428] T 28b608d704d8479c9387a6a9732493f5 P 5bbd2f6ea420477192319afe8301bcf4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 20:31:50.843138 6706 ts_tablet_manager.cc:1918] T a0c93605976349d18aa9a31c3e91a48e P e372f443fb0b481d9ca913ba70fb63c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:31:50.843251 6706 log.cc:1196] T a0c93605976349d18aa9a31c3e91a48e P e372f443fb0b481d9ca913ba70fb63c2: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-0/wal/wals/a0c93605976349d18aa9a31c3e91a48e
I20260619 20:31:50.843518 6706 ts_tablet_manager.cc:1939] T a0c93605976349d18aa9a31c3e91a48e P e372f443fb0b481d9ca913ba70fb63c2: Deleting consensus metadata
I20260619 20:31:50.844161 6146 catalog_manager.cc:4841] TS e372f443fb0b481d9ca913ba70fb63c2 (127.4.83.65:45613): tablet a0c93605976349d18aa9a31c3e91a48e (table test-table [id=c9c32ba644ed426bb4f700c20d439597]) successfully deleted
I20260619 20:31:51.025588 6673 raft_consensus.cc:482] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:31:51.025697 6673 raft_consensus.cc:504] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:51.026073 6673 leader_election.cc:288] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7482052ce1824e138d8b6e9b5b48e24d (127.4.83.66:40339), 5bbd2f6ea420477192319afe8301bcf4 (127.4.83.67:37647)
I20260619 20:31:51.029745 6663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:51.026419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:45115 (local address 127.4.83.67:37647)
0619 20:31:51.026477 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:51.026479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:51.026711 (+ 232us) server_negotiation.cc:407] Connection header received
0619 20:31:51.026760 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:51.026761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:51.026773 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:51.026834 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:31:51.026989 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:51.027743 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:51.028995 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:51.029398 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:51.029461 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:51.029473 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:51.029618 (+ 145us) server_negotiation.cc:299] Negotiation successful
0619 20:31:51.029665 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 20:31:51.029834 6712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:51.026289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:40339 (local address 127.4.83.65:59963)
0619 20:31:51.026423 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:51.026435 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:51.026507 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:51.026739 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:51.026740 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:51.026742 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:31:51.026795 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:51.026797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:51.027537 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:51.027539 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:51.028774 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:51.028777 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:51.029577 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:51.029578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:51.029650 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:51.029653 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:31:51.029710 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 20:31:51.029759 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":64,"threads_started":1}
I20260619 20:31:51.029891 6664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:51.026519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:59963 (local address 127.4.83.66:40339)
0619 20:31:51.026565 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:51.026566 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:51.026574 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:31:51.026604 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:51.026605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:51.026617 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:51.026679 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:31:51.026873 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:51.027448 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:51.028897 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:51.029488 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:51.029584 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:51.029598 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:51.029725 (+ 127us) server_negotiation.cc:299] Negotiation successful
0619 20:31:51.029793 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 20:31:51.030107 6713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:51.026342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:37647 (local address 127.4.83.65:45115)
0619 20:31:51.026601 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:51.026611 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:51.026668 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:51.026863 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:51.026864 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:51.026866 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:31:51.026917 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:51.026919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:51.027885 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:51.027885 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:51.028910 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:51.028911 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:51.029472 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:51.029473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:51.029533 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:51.029537 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:31:51.029958 (+ 421us) client_negotiation.cc:240] Negotiation successful
0619 20:31:51.029998 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":93,"threads_started":1}
I20260619 20:31:51.030143 6456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28b608d704d8479c9387a6a9732493f5" candidate_uuid: "e372f443fb0b481d9ca913ba70fb63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7482052ce1824e138d8b6e9b5b48e24d" is_pre_election: true
I20260619 20:31:51.030243 6456 raft_consensus.cc:2454] T 28b608d704d8479c9387a6a9732493f5 P 7482052ce1824e138d8b6e9b5b48e24d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e372f443fb0b481d9ca913ba70fb63c2 in term 0.
I20260619 20:31:51.030395 6596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28b608d704d8479c9387a6a9732493f5" candidate_uuid: "e372f443fb0b481d9ca913ba70fb63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bbd2f6ea420477192319afe8301bcf4" is_pre_election: true
I20260619 20:31:51.030457 6247 leader_election.cc:302] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7482052ce1824e138d8b6e9b5b48e24d, e372f443fb0b481d9ca913ba70fb63c2; no voters:
I20260619 20:31:51.030547 6596 raft_consensus.cc:2454] T 28b608d704d8479c9387a6a9732493f5 P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e372f443fb0b481d9ca913ba70fb63c2 in term 0.
I20260619 20:31:51.030643 6673 raft_consensus.cc:2781] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:31:51.030683 6673 raft_consensus.cc:482] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:31:51.030701 6673 raft_consensus.cc:3037] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:51.031803 6673 raft_consensus.cc:504] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:51.032068 6673 leader_election.cc:288] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2 [CANDIDATE]: Term 1 election: Requested vote from peers 7482052ce1824e138d8b6e9b5b48e24d (127.4.83.66:40339), 5bbd2f6ea420477192319afe8301bcf4 (127.4.83.67:37647)
I20260619 20:31:51.032281 6456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28b608d704d8479c9387a6a9732493f5" candidate_uuid: "e372f443fb0b481d9ca913ba70fb63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7482052ce1824e138d8b6e9b5b48e24d"
I20260619 20:31:51.032299 6596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28b608d704d8479c9387a6a9732493f5" candidate_uuid: "e372f443fb0b481d9ca913ba70fb63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bbd2f6ea420477192319afe8301bcf4"
I20260619 20:31:51.032382 6456 raft_consensus.cc:3037] T 28b608d704d8479c9387a6a9732493f5 P 7482052ce1824e138d8b6e9b5b48e24d [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:51.032405 6596 raft_consensus.cc:3037] T 28b608d704d8479c9387a6a9732493f5 P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:51.033205 6596 raft_consensus.cc:2454] T 28b608d704d8479c9387a6a9732493f5 P 5bbd2f6ea420477192319afe8301bcf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e372f443fb0b481d9ca913ba70fb63c2 in term 1.
I20260619 20:31:51.033275 6456 raft_consensus.cc:2454] T 28b608d704d8479c9387a6a9732493f5 P 7482052ce1824e138d8b6e9b5b48e24d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e372f443fb0b481d9ca913ba70fb63c2 in term 1.
I20260619 20:31:51.033388 6248 leader_election.cc:302] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5bbd2f6ea420477192319afe8301bcf4, e372f443fb0b481d9ca913ba70fb63c2; no voters:
I20260619 20:31:51.033496 6673 raft_consensus.cc:2781] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:31:51.033703 6673 raft_consensus.cc:686] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2 [term 1 LEADER]: Becoming Leader. State: Replica: e372f443fb0b481d9ca913ba70fb63c2, State: Running, Role: LEADER
I20260619 20:31:51.033795 6673 consensus_queue.cc:237] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:51.034786 6158 catalog_manager.cc:5495] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to e372f443fb0b481d9ca913ba70fb63c2 (127.4.83.65). New cstate: current_term: 1 leader_uuid: "e372f443fb0b481d9ca913ba70fb63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } health_report { overall_health: UNKNOWN } } }
I20260619 20:31:51.071265 6716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:51.068325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46736 (local address 127.4.83.65:45613)
0619 20:31:51.068480 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:51.068482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:51.068493 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:31:51.068582 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:51.068582 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:51.068605 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:51.068675 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:31:51.068867 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:51.069422 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:51.070328 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:51.070548 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:51.070616 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:51.070851 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:31:51.070975 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:31:51.071049 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:51.071160 (+ 111us) server_negotiation.cc:299] Negotiation successful
0619 20:31:51.071183 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":92,"threads_started":1}
I20260619 20:31:51.077585 6719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:51.074975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:37647 (local address 127.4.83.65:60521)
0619 20:31:51.075140 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:51.075152 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:51.075216 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:51.075335 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:51.075336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:51.075338 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:31:51.075407 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:51.075408 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:51.076061 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:51.076062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:51.076658 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:51.076660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:51.077026 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:51.077027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:51.077076 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:51.077095 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:31:51.077391 (+ 296us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:31:51.077393 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:31:51.077442 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 20:31:51.077497 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260619 20:31:51.077562 6663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:51.075101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:60521 (local address 127.4.83.67:37647)
0619 20:31:51.075172 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:51.075173 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:51.075200 (+ 27us) server_negotiation.cc:407] Connection header received
0619 20:31:51.075276 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:51.075277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:51.075287 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:51.075334 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:31:51.075459 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:51.076000 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:51.076736 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:51.076939 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:51.076993 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:51.077292 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:31:51.077337 (+ 45us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:31:51.077382 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:51.077446 (+ 64us) server_negotiation.cc:299] Negotiation successful
0619 20:31:51.077472 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 20:31:51.080437 6456 raft_consensus.cc:1261] T 28b608d704d8479c9387a6a9732493f5 P 7482052ce1824e138d8b6e9b5b48e24d [term 1 FOLLOWER]: Refusing update from remote peer e372f443fb0b481d9ca913ba70fb63c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:31:51.080534 6596 raft_consensus.cc:1261] T 28b608d704d8479c9387a6a9732493f5 P 5bbd2f6ea420477192319afe8301bcf4 [term 1 FOLLOWER]: Refusing update from remote peer e372f443fb0b481d9ca913ba70fb63c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:31:51.080747 6714 consensus_queue.cc:1035] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:31:51.080950 6673 consensus_queue.cc:1035] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:31:51.089231 6716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:51.086856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:45567 (local address 127.4.83.65:45613)
0619 20:31:51.086926 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:51.086928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:51.086938 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:31:51.086966 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:51.086967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:51.086979 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:51.087036 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:31:51.087269 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:51.087863 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:51.088565 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:51.088807 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:51.088867 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:51.088988 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:31:51.089022 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:31:51.089063 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:51.089123 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 20:31:51.089146 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 20:31:51.089232 6648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:51.086772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:45613 (local address 127.4.83.67:45567)
0619 20:31:51.086827 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:51.086840 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:51.086886 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:51.087103 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:51.087105 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:51.087106 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:31:51.087158 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:51.087160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:51.087938 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:51.087939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:51.088491 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:51.088493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:51.088871 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:51.088873 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:51.088927 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:51.088931 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:31:51.089070 (+ 139us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:31:51.089072 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:31:51.089118 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 20:31:51.089163 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 20:31:51.089605 6691 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 20:31:52.620548 6159 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53488:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:31:52.620708 6159 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53488:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:31:52.621863 6159 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1164ce5a83bc48bc91a00bac7c0d9552: Sending DeleteTablet for 3 replicas of tablet 28b608d704d8479c9387a6a9732493f5
I20260619 20:31:52.622174 6296 tablet_service.cc:1514] Processing DeleteTablet for tablet 28b608d704d8479c9387a6a9732493f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:31:52 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:46734
I20260619 20:31:52.622437 6746 tablet_replica.cc:330] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2: stopping tablet replica
I20260619 20:31:52.622459 6436 tablet_service.cc:1514] Processing DeleteTablet for tablet 28b608d704d8479c9387a6a9732493f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:31:52 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:50352
I20260619 20:31:52.622551 6746 raft_consensus.cc:2229] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:31:52.622700 6746 raft_consensus.cc:2258] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:31:52.622682 6575 tablet_service.cc:1514] Processing DeleteTablet for tablet 28b608d704d8479c9387a6a9732493f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:31:52 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:51444
I20260619 20:31:52.622879 6748 tablet_replica.cc:330] T 28b608d704d8479c9387a6a9732493f5 P 5bbd2f6ea420477192319afe8301bcf4: stopping tablet replica
I20260619 20:31:52.622939 6748 raft_consensus.cc:2229] T 28b608d704d8479c9387a6a9732493f5 P 5bbd2f6ea420477192319afe8301bcf4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:31:52.622979 6748 raft_consensus.cc:2258] T 28b608d704d8479c9387a6a9732493f5 P 5bbd2f6ea420477192319afe8301bcf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:31:52.623384 6748 ts_tablet_manager.cc:1905] T 28b608d704d8479c9387a6a9732493f5 P 5bbd2f6ea420477192319afe8301bcf4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:31:52.624910 6747 tablet_replica.cc:330] T 28b608d704d8479c9387a6a9732493f5 P 7482052ce1824e138d8b6e9b5b48e24d: stopping tablet replica
I20260619 20:31:52.625007 6747 raft_consensus.cc:2229] T 28b608d704d8479c9387a6a9732493f5 P 7482052ce1824e138d8b6e9b5b48e24d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:31:52.625065 6747 raft_consensus.cc:2258] T 28b608d704d8479c9387a6a9732493f5 P 7482052ce1824e138d8b6e9b5b48e24d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:31:52.625365 6747 ts_tablet_manager.cc:1905] T 28b608d704d8479c9387a6a9732493f5 P 7482052ce1824e138d8b6e9b5b48e24d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:31:52.625813 6748 ts_tablet_manager.cc:1918] T 28b608d704d8479c9387a6a9732493f5 P 5bbd2f6ea420477192319afe8301bcf4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 20:31:52.625833 6746 ts_tablet_manager.cc:1905] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:31:52.625881 6748 log.cc:1196] T 28b608d704d8479c9387a6a9732493f5 P 5bbd2f6ea420477192319afe8301bcf4: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-2/wal/wals/28b608d704d8479c9387a6a9732493f5
I20260619 20:31:52.626219 6748 ts_tablet_manager.cc:1939] T 28b608d704d8479c9387a6a9732493f5 P 5bbd2f6ea420477192319afe8301bcf4: Deleting consensus metadata
I20260619 20:31:52.626693 6144 catalog_manager.cc:4841] TS 5bbd2f6ea420477192319afe8301bcf4 (127.4.83.67:37647): tablet 28b608d704d8479c9387a6a9732493f5 (table test-table [id=f63f664ef41b4055ae3defcc59ea3212]) successfully deleted
I20260619 20:31:52.627197 6747 ts_tablet_manager.cc:1918] T 28b608d704d8479c9387a6a9732493f5 P 7482052ce1824e138d8b6e9b5b48e24d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 20:31:52.627252 6747 log.cc:1196] T 28b608d704d8479c9387a6a9732493f5 P 7482052ce1824e138d8b6e9b5b48e24d: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-1/wal/wals/28b608d704d8479c9387a6a9732493f5
I20260619 20:31:52.627436 6747 ts_tablet_manager.cc:1939] T 28b608d704d8479c9387a6a9732493f5 P 7482052ce1824e138d8b6e9b5b48e24d: Deleting consensus metadata
I20260619 20:31:52.627496 6746 ts_tablet_manager.cc:1918] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 20:31:52.627583 6746 log.cc:1196] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-0/wal/wals/28b608d704d8479c9387a6a9732493f5
I20260619 20:31:52.627844 6746 ts_tablet_manager.cc:1939] T 28b608d704d8479c9387a6a9732493f5 P e372f443fb0b481d9ca913ba70fb63c2: Deleting consensus metadata
I20260619 20:31:52.627928 6143 catalog_manager.cc:4841] TS 7482052ce1824e138d8b6e9b5b48e24d (127.4.83.66:40339): tablet 28b608d704d8479c9387a6a9732493f5 (table test-table [id=f63f664ef41b4055ae3defcc59ea3212]) successfully deleted
I20260619 20:31:52.628154 6146 catalog_manager.cc:4841] TS e372f443fb0b481d9ca913ba70fb63c2 (127.4.83.65:45613): tablet 28b608d704d8479c9387a6a9732493f5 (table test-table [id=f63f664ef41b4055ae3defcc59ea3212]) successfully deleted
Jun 19 20:31:52 dist-test-slave-4m58 krb5kdc[6114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901112, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 20:31:52 dist-test-slave-4m58 krb5kdc[6114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901112, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:31:52.657128 6756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:52.644894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53508 (local address 127.4.83.126:44167)
0619 20:31:52.645131 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:52.645133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:52.645142 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:31:52.645175 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:52.645175 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:52.645189 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:52.645261 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:52.645952 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:52.646533 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:52.647447 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:52.647723 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:52.647798 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:52.654388 (+ 6590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:52.654390 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:52.654391 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:52.654403 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:52.656063 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:52.656434 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:52.656435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:52.656436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:52.656490 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:52.656632 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:52.656632 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:52.656633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:52.656769 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:52.656852 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:52.656932 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 20:31:52.657021 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":148,"threads_started":1}
I20260619 20:31:52.658783 6159 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53508:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:31:52.658964 6159 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:31:52.660354 6436 tablet_service.cc:1467] Processing CreateTablet for tablet b23a1a1f056e425d89a08ffa7167cc4f (DEFAULT_TABLE table=test-table [id=e8b85448318a47f1bd863d60165ac5c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:31:52.660372 6575 tablet_service.cc:1467] Processing CreateTablet for tablet b23a1a1f056e425d89a08ffa7167cc4f (DEFAULT_TABLE table=test-table [id=e8b85448318a47f1bd863d60165ac5c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:31:52.660539 6575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b23a1a1f056e425d89a08ffa7167cc4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:52.660544 6436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b23a1a1f056e425d89a08ffa7167cc4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:52.661787 6758 tablet_bootstrap.cc:492] T b23a1a1f056e425d89a08ffa7167cc4f P 5bbd2f6ea420477192319afe8301bcf4: Bootstrap starting.
I20260619 20:31:52.661866 6296 tablet_service.cc:1467] Processing CreateTablet for tablet b23a1a1f056e425d89a08ffa7167cc4f (DEFAULT_TABLE table=test-table [id=e8b85448318a47f1bd863d60165ac5c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:31:52.662014 6296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b23a1a1f056e425d89a08ffa7167cc4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:52.662165 6757 tablet_bootstrap.cc:492] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d: Bootstrap starting.
I20260619 20:31:52.662235 6758 tablet_bootstrap.cc:654] T b23a1a1f056e425d89a08ffa7167cc4f P 5bbd2f6ea420477192319afe8301bcf4: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:52.662587 6757 tablet_bootstrap.cc:654] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:52.662976 6758 tablet_bootstrap.cc:492] T b23a1a1f056e425d89a08ffa7167cc4f P 5bbd2f6ea420477192319afe8301bcf4: No bootstrap required, opened a new log
I20260619 20:31:52.663031 6758 ts_tablet_manager.cc:1397] T b23a1a1f056e425d89a08ffa7167cc4f P 5bbd2f6ea420477192319afe8301bcf4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:31:52.663192 6757 tablet_bootstrap.cc:492] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d: No bootstrap required, opened a new log
I20260619 20:31:52.663233 6757 ts_tablet_manager.cc:1397] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:31:52.663173 6758 raft_consensus.cc:348] T b23a1a1f056e425d89a08ffa7167cc4f P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:52.663285 6758 raft_consensus.cc:374] T b23a1a1f056e425d89a08ffa7167cc4f P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:52.663321 6758 raft_consensus.cc:729] T b23a1a1f056e425d89a08ffa7167cc4f P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bbd2f6ea420477192319afe8301bcf4, State: Initialized, Role: FOLLOWER
I20260619 20:31:52.663373 6758 consensus_queue.cc:260] T b23a1a1f056e425d89a08ffa7167cc4f P 5bbd2f6ea420477192319afe8301bcf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:52.663388 6757 raft_consensus.cc:348] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:52.663461 6757 raft_consensus.cc:374] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:52.663481 6757 raft_consensus.cc:729] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7482052ce1824e138d8b6e9b5b48e24d, State: Initialized, Role: FOLLOWER
I20260619 20:31:52.663542 6757 consensus_queue.cc:260] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:52.663646 6758 ts_tablet_manager.cc:1428] T b23a1a1f056e425d89a08ffa7167cc4f P 5bbd2f6ea420477192319afe8301bcf4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:31:52.663864 6757 ts_tablet_manager.cc:1428] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:31:52.663980 6761 tablet_bootstrap.cc:492] T b23a1a1f056e425d89a08ffa7167cc4f P e372f443fb0b481d9ca913ba70fb63c2: Bootstrap starting.
I20260619 20:31:52.664440 6761 tablet_bootstrap.cc:654] T b23a1a1f056e425d89a08ffa7167cc4f P e372f443fb0b481d9ca913ba70fb63c2: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:52.665081 6761 tablet_bootstrap.cc:492] T b23a1a1f056e425d89a08ffa7167cc4f P e372f443fb0b481d9ca913ba70fb63c2: No bootstrap required, opened a new log
I20260619 20:31:52.665127 6761 ts_tablet_manager.cc:1397] T b23a1a1f056e425d89a08ffa7167cc4f P e372f443fb0b481d9ca913ba70fb63c2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:31:52.665236 6761 raft_consensus.cc:348] T b23a1a1f056e425d89a08ffa7167cc4f P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:52.665318 6761 raft_consensus.cc:374] T b23a1a1f056e425d89a08ffa7167cc4f P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:52.665337 6761 raft_consensus.cc:729] T b23a1a1f056e425d89a08ffa7167cc4f P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e372f443fb0b481d9ca913ba70fb63c2, State: Initialized, Role: FOLLOWER
I20260619 20:31:52.665395 6761 consensus_queue.cc:260] T b23a1a1f056e425d89a08ffa7167cc4f P e372f443fb0b481d9ca913ba70fb63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:52.665498 6761 ts_tablet_manager.cc:1428] T b23a1a1f056e425d89a08ffa7167cc4f P e372f443fb0b481d9ca913ba70fb63c2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:52.810230 6762 raft_consensus.cc:482] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:31:52.810395 6762 raft_consensus.cc:504] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:52.811038 6762 leader_election.cc:288] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e372f443fb0b481d9ca913ba70fb63c2 (127.4.83.65:45613), 5bbd2f6ea420477192319afe8301bcf4 (127.4.83.67:37647)
I20260619 20:31:52.815400 6766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:52.811237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:33065 (local address 127.4.83.65:45613)
0619 20:31:52.811413 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:52.811415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:52.811475 (+ 60us) server_negotiation.cc:407] Connection header received
0619 20:31:52.811540 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:52.811541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:52.811554 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:52.811642 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:31:52.811938 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:52.812748 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:52.814291 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:52.814955 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:52.815035 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:52.815053 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:52.815255 (+ 202us) server_negotiation.cc:299] Negotiation successful
0619 20:31:52.815306 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":102,"threads_started":1}
I20260619 20:31:52.815400 6767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:52.811233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:48065 (local address 127.4.83.67:37647)
0619 20:31:52.811477 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:52.811479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:52.811487 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:31:52.811540 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:52.811541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:52.811554 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:52.811649 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:31:52.811923 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:52.812764 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:52.814368 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:52.814961 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:52.815035 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:52.815053 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:52.815252 (+ 199us) server_negotiation.cc:299] Negotiation successful
0619 20:31:52.815296 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":109,"threads_started":1}
I20260619 20:31:52.815423 6764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:52.811021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:37647 (local address 127.4.83.66:48065)
0619 20:31:52.811401 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:52.811412 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:52.811484 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:52.811696 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:52.811698 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:52.811700 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:31:52.811824 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:52.811853 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:52.812827 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:52.812828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:52.814283 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:52.814284 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:52.815085 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:52.815085 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:52.815159 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:52.815166 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 20:31:52.815240 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 20:31:52.815300 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":65,"thread_start_us":122,"threads_started":1}
I20260619 20:31:52.815431 6765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:52.811021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:45613 (local address 127.4.83.66:33065)
0619 20:31:52.811366 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:52.811381 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:52.811473 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:52.811690 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:52.811692 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:52.811695 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:31:52.811824 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:52.811853 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:52.812828 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:52.812829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:52.814184 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:52.814187 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:52.815065 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:52.815067 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:52.815159 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:52.815191 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 20:31:52.815243 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 20:31:52.815300 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":10,"thread_start_us":165,"threads_started":1}
I20260619 20:31:52.815930 6316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b23a1a1f056e425d89a08ffa7167cc4f" candidate_uuid: "7482052ce1824e138d8b6e9b5b48e24d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e372f443fb0b481d9ca913ba70fb63c2" is_pre_election: true
I20260619 20:31:52.816036 6316 raft_consensus.cc:2454] T b23a1a1f056e425d89a08ffa7167cc4f P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7482052ce1824e138d8b6e9b5b48e24d in term 0.
I20260619 20:31:52.816227 6596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b23a1a1f056e425d89a08ffa7167cc4f" candidate_uuid: "7482052ce1824e138d8b6e9b5b48e24d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bbd2f6ea420477192319afe8301bcf4" is_pre_election: true
I20260619 20:31:52.816267 6390 leader_election.cc:302] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7482052ce1824e138d8b6e9b5b48e24d, e372f443fb0b481d9ca913ba70fb63c2; no voters:
I20260619 20:31:52.816324 6596 raft_consensus.cc:2454] T b23a1a1f056e425d89a08ffa7167cc4f P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7482052ce1824e138d8b6e9b5b48e24d in term 0.
I20260619 20:31:52.816435 6762 raft_consensus.cc:2781] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:31:52.816491 6762 raft_consensus.cc:482] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:31:52.816514 6762 raft_consensus.cc:3037] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:52.817365 6762 raft_consensus.cc:504] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:52.817518 6762 leader_election.cc:288] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d [CANDIDATE]: Term 1 election: Requested vote from peers e372f443fb0b481d9ca913ba70fb63c2 (127.4.83.65:45613), 5bbd2f6ea420477192319afe8301bcf4 (127.4.83.67:37647)
I20260619 20:31:52.817713 6596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b23a1a1f056e425d89a08ffa7167cc4f" candidate_uuid: "7482052ce1824e138d8b6e9b5b48e24d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bbd2f6ea420477192319afe8301bcf4"
I20260619 20:31:52.817703 6316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b23a1a1f056e425d89a08ffa7167cc4f" candidate_uuid: "7482052ce1824e138d8b6e9b5b48e24d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e372f443fb0b481d9ca913ba70fb63c2"
I20260619 20:31:52.817788 6596 raft_consensus.cc:3037] T b23a1a1f056e425d89a08ffa7167cc4f P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:52.817791 6316 raft_consensus.cc:3037] T b23a1a1f056e425d89a08ffa7167cc4f P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:52.818501 6596 raft_consensus.cc:2454] T b23a1a1f056e425d89a08ffa7167cc4f P 5bbd2f6ea420477192319afe8301bcf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7482052ce1824e138d8b6e9b5b48e24d in term 1.
I20260619 20:31:52.818621 6316 raft_consensus.cc:2454] T b23a1a1f056e425d89a08ffa7167cc4f P e372f443fb0b481d9ca913ba70fb63c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7482052ce1824e138d8b6e9b5b48e24d in term 1.
I20260619 20:31:52.818699 6388 leader_election.cc:302] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5bbd2f6ea420477192319afe8301bcf4, 7482052ce1824e138d8b6e9b5b48e24d; no voters:
I20260619 20:31:52.818807 6762 raft_consensus.cc:2781] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:31:52.819008 6762 raft_consensus.cc:686] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d [term 1 LEADER]: Becoming Leader. State: Replica: 7482052ce1824e138d8b6e9b5b48e24d, State: Running, Role: LEADER
I20260619 20:31:52.819087 6762 consensus_queue.cc:237] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:52.819974 6158 catalog_manager.cc:5495] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7482052ce1824e138d8b6e9b5b48e24d (127.4.83.66). New cstate: current_term: 1 leader_uuid: "7482052ce1824e138d8b6e9b5b48e24d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } health_report { overall_health: UNKNOWN } } }
I20260619 20:31:52.854839 6771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:52.852169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50358 (local address 127.4.83.66:40339)
0619 20:31:52.852345 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:52.852346 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:52.852352 (+ 6us) server_negotiation.cc:407] Connection header received
0619 20:31:52.852393 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:52.852394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:52.852418 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:52.852493 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:31:52.852669 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:52.853197 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:52.854004 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:52.854234 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:52.854299 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:52.854462 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:31:52.854599 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:31:52.854662 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:52.854715 (+ 53us) server_negotiation.cc:299] Negotiation successful
0619 20:31:52.854736 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":116,"threads_started":1}
I20260619 20:31:52.861846 6774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:52.858467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:37647 (local address 127.4.83.66:45433)
0619 20:31:52.858868 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:52.858882 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:52.858941 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:52.859274 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:52.859275 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:52.859327 (+ 52us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:31:52.859421 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:52.859425 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:52.860226 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:52.860227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:52.860900 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:52.860902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:52.861362 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:52.861363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:52.861434 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:52.861462 (+ 28us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:31:52.861674 (+ 212us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:31:52.861675 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 20:31:52.861706 (+ 31us) client_negotiation.cc:240] Negotiation successful
0619 20:31:52.861762 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":139,"threads_started":1}
I20260619 20:31:52.861833 6767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:52.858798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:45433 (local address 127.4.83.67:37647)
0619 20:31:52.858913 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:52.858915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:52.858953 (+ 38us) server_negotiation.cc:407] Connection header received
0619 20:31:52.859151 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:52.859152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:52.859170 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:52.859233 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:31:52.859551 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:52.860134 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:52.861049 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:52.861269 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:52.861336 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:52.861566 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:31:52.861604 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:31:52.861672 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:52.861710 (+ 38us) server_negotiation.cc:299] Negotiation successful
0619 20:31:52.861738 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260619 20:31:52.864434 6596 raft_consensus.cc:1261] T b23a1a1f056e425d89a08ffa7167cc4f P 5bbd2f6ea420477192319afe8301bcf4 [term 1 FOLLOWER]: Refusing update from remote peer 7482052ce1824e138d8b6e9b5b48e24d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:31:52.864434 6316 raft_consensus.cc:1261] T b23a1a1f056e425d89a08ffa7167cc4f P e372f443fb0b481d9ca913ba70fb63c2 [term 1 FOLLOWER]: Refusing update from remote peer 7482052ce1824e138d8b6e9b5b48e24d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:31:52.864718 6768 consensus_queue.cc:1035] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:31:52.864863 6762 consensus_queue.cc:1035] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:31:52.872859 6771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:52.870199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:46177 (local address 127.4.83.66:40339)
0619 20:31:52.870239 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:52.870241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:52.870349 (+ 108us) server_negotiation.cc:407] Connection header received
0619 20:31:52.870404 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:52.870405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:52.870419 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:52.870472 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:31:52.870621 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:52.871161 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:52.872059 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:52.872332 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:52.872392 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:52.872597 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:31:52.872636 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:31:52.872702 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:52.872762 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 20:31:52.872784 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 20:31:52.872874 6779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:52.870160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:40339 (local address 127.4.83.67:46177)
0619 20:31:52.870283 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:52.870294 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:52.870356 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:52.870473 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:52.870474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:52.870476 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:31:52.870548 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:52.870550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:52.871265 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:52.871266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:52.871963 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:52.871967 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:52.872399 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:52.872400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:52.872468 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:52.872476 (+ 8us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:31:52.872713 (+ 237us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:31:52.872715 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:31:52.872757 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 20:31:52.872802 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":77,"threads_started":1}
I20260619 20:31:52.883890 6159 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:31:52.884044 6159 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:31:52.884949 6159 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1164ce5a83bc48bc91a00bac7c0d9552: Sending DeleteTablet for 3 replicas of tablet b23a1a1f056e425d89a08ffa7167cc4f
I20260619 20:31:52.885279 6436 tablet_service.cc:1514] Processing DeleteTablet for tablet b23a1a1f056e425d89a08ffa7167cc4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:31:52 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:50352
I20260619 20:31:52.885288 6296 tablet_service.cc:1514] Processing DeleteTablet for tablet b23a1a1f056e425d89a08ffa7167cc4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:31:52 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:46734
I20260619 20:31:52.885387 6747 tablet_replica.cc:330] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d: stopping tablet replica
I20260619 20:31:52.885388 6746 tablet_replica.cc:330] T b23a1a1f056e425d89a08ffa7167cc4f P e372f443fb0b481d9ca913ba70fb63c2: stopping tablet replica
I20260619 20:31:52.885370 6575 tablet_service.cc:1514] Processing DeleteTablet for tablet b23a1a1f056e425d89a08ffa7167cc4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:31:52 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:51444
I20260619 20:31:52.885439 6747 raft_consensus.cc:2229] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:31:52.885447 6746 raft_consensus.cc:2229] T b23a1a1f056e425d89a08ffa7167cc4f P e372f443fb0b481d9ca913ba70fb63c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:31:52.885489 6746 raft_consensus.cc:2258] T b23a1a1f056e425d89a08ffa7167cc4f P e372f443fb0b481d9ca913ba70fb63c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:31:52.885548 6747 raft_consensus.cc:2258] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:31:52.885588 6748 tablet_replica.cc:330] T b23a1a1f056e425d89a08ffa7167cc4f P 5bbd2f6ea420477192319afe8301bcf4: stopping tablet replica
I20260619 20:31:52.885660 6748 raft_consensus.cc:2229] T b23a1a1f056e425d89a08ffa7167cc4f P 5bbd2f6ea420477192319afe8301bcf4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:31:52.885710 6748 raft_consensus.cc:2258] T b23a1a1f056e425d89a08ffa7167cc4f P 5bbd2f6ea420477192319afe8301bcf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:31:52.885933 6747 ts_tablet_manager.cc:1905] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:31:52.886186 6746 ts_tablet_manager.cc:1905] T b23a1a1f056e425d89a08ffa7167cc4f P e372f443fb0b481d9ca913ba70fb63c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:31:52.888012 6746 ts_tablet_manager.cc:1918] T b23a1a1f056e425d89a08ffa7167cc4f P e372f443fb0b481d9ca913ba70fb63c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:31:52.888012 6747 ts_tablet_manager.cc:1918] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:31:52.888083 6747 log.cc:1196] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-1/wal/wals/b23a1a1f056e425d89a08ffa7167cc4f
I20260619 20:31:52.888083 6746 log.cc:1196] T b23a1a1f056e425d89a08ffa7167cc4f P e372f443fb0b481d9ca913ba70fb63c2: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-0/wal/wals/b23a1a1f056e425d89a08ffa7167cc4f
I20260619 20:31:52.888301 6746 ts_tablet_manager.cc:1939] T b23a1a1f056e425d89a08ffa7167cc4f P e372f443fb0b481d9ca913ba70fb63c2: Deleting consensus metadata
I20260619 20:31:52.888301 6747 ts_tablet_manager.cc:1939] T b23a1a1f056e425d89a08ffa7167cc4f P 7482052ce1824e138d8b6e9b5b48e24d: Deleting consensus metadata
I20260619 20:31:52.888796 6143 catalog_manager.cc:4841] TS 7482052ce1824e138d8b6e9b5b48e24d (127.4.83.66:40339): tablet b23a1a1f056e425d89a08ffa7167cc4f (table test-table [id=e8b85448318a47f1bd863d60165ac5c6]) successfully deleted
I20260619 20:31:52.888803 6146 catalog_manager.cc:4841] TS e372f443fb0b481d9ca913ba70fb63c2 (127.4.83.65:45613): tablet b23a1a1f056e425d89a08ffa7167cc4f (table test-table [id=e8b85448318a47f1bd863d60165ac5c6]) successfully deleted
I20260619 20:31:52.888921 6748 ts_tablet_manager.cc:1905] T b23a1a1f056e425d89a08ffa7167cc4f P 5bbd2f6ea420477192319afe8301bcf4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:31:52.888976 6159 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53508:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:31:52.889189 6159 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:31:52.890811 6748 ts_tablet_manager.cc:1918] T b23a1a1f056e425d89a08ffa7167cc4f P 5bbd2f6ea420477192319afe8301bcf4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:31:52.890902 6748 log.cc:1196] T b23a1a1f056e425d89a08ffa7167cc4f P 5bbd2f6ea420477192319afe8301bcf4: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-2/wal/wals/b23a1a1f056e425d89a08ffa7167cc4f
I20260619 20:31:52.890879 6436 tablet_service.cc:1467] Processing CreateTablet for tablet 1dc6a0ba647a4ea5879c1e0cfb2e4847 (DEFAULT_TABLE table=test-table [id=9e9ecaf8a76b4a9281b90964c6b3f352]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:31:52.891052 6436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dc6a0ba647a4ea5879c1e0cfb2e4847. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:52.891115 6748 ts_tablet_manager.cc:1939] T b23a1a1f056e425d89a08ffa7167cc4f P 5bbd2f6ea420477192319afe8301bcf4: Deleting consensus metadata
I20260619 20:31:52.891431 6296 tablet_service.cc:1467] Processing CreateTablet for tablet 1dc6a0ba647a4ea5879c1e0cfb2e4847 (DEFAULT_TABLE table=test-table [id=9e9ecaf8a76b4a9281b90964c6b3f352]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:31:52.891573 6296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dc6a0ba647a4ea5879c1e0cfb2e4847. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:52.892616 6757 tablet_bootstrap.cc:492] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 7482052ce1824e138d8b6e9b5b48e24d: Bootstrap starting.
I20260619 20:31:52.892918 6144 catalog_manager.cc:4841] TS 5bbd2f6ea420477192319afe8301bcf4 (127.4.83.67:37647): tablet b23a1a1f056e425d89a08ffa7167cc4f (table test-table [id=e8b85448318a47f1bd863d60165ac5c6]) successfully deleted
I20260619 20:31:52.893076 6575 tablet_service.cc:1467] Processing CreateTablet for tablet 1dc6a0ba647a4ea5879c1e0cfb2e4847 (DEFAULT_TABLE table=test-table [id=9e9ecaf8a76b4a9281b90964c6b3f352]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:31:52.893153 6757 tablet_bootstrap.cc:654] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 7482052ce1824e138d8b6e9b5b48e24d: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:52.893188 6575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dc6a0ba647a4ea5879c1e0cfb2e4847. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:52.893234 6761 tablet_bootstrap.cc:492] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P e372f443fb0b481d9ca913ba70fb63c2: Bootstrap starting.
I20260619 20:31:52.893594 6761 tablet_bootstrap.cc:654] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P e372f443fb0b481d9ca913ba70fb63c2: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:52.894479 6757 tablet_bootstrap.cc:492] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 7482052ce1824e138d8b6e9b5b48e24d: No bootstrap required, opened a new log
I20260619 20:31:52.894539 6757 ts_tablet_manager.cc:1397] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 7482052ce1824e138d8b6e9b5b48e24d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:31:52.894758 6757 raft_consensus.cc:348] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 7482052ce1824e138d8b6e9b5b48e24d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:52.894840 6757 raft_consensus.cc:374] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 7482052ce1824e138d8b6e9b5b48e24d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:52.894861 6757 raft_consensus.cc:729] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 7482052ce1824e138d8b6e9b5b48e24d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7482052ce1824e138d8b6e9b5b48e24d, State: Initialized, Role: FOLLOWER
I20260619 20:31:52.894919 6757 consensus_queue.cc:260] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 7482052ce1824e138d8b6e9b5b48e24d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:52.895040 6757 ts_tablet_manager.cc:1428] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 7482052ce1824e138d8b6e9b5b48e24d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:52.895190 6758 tablet_bootstrap.cc:492] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 5bbd2f6ea420477192319afe8301bcf4: Bootstrap starting.
I20260619 20:31:52.895563 6758 tablet_bootstrap.cc:654] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 5bbd2f6ea420477192319afe8301bcf4: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:52.896196 6758 tablet_bootstrap.cc:492] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 5bbd2f6ea420477192319afe8301bcf4: No bootstrap required, opened a new log
I20260619 20:31:52.896250 6758 ts_tablet_manager.cc:1397] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 5bbd2f6ea420477192319afe8301bcf4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:31:52.896360 6758 raft_consensus.cc:348] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:52.896425 6758 raft_consensus.cc:374] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:52.896437 6758 raft_consensus.cc:729] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bbd2f6ea420477192319afe8301bcf4, State: Initialized, Role: FOLLOWER
I20260619 20:31:52.896476 6758 consensus_queue.cc:260] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 5bbd2f6ea420477192319afe8301bcf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:52.896569 6758 ts_tablet_manager.cc:1428] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 5bbd2f6ea420477192319afe8301bcf4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 20:31:52.896927 6761 tablet_bootstrap.cc:492] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P e372f443fb0b481d9ca913ba70fb63c2: No bootstrap required, opened a new log
I20260619 20:31:52.896992 6761 ts_tablet_manager.cc:1397] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P e372f443fb0b481d9ca913ba70fb63c2: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 20:31:52.897169 6761 raft_consensus.cc:348] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:52.897260 6761 raft_consensus.cc:374] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:52.897286 6761 raft_consensus.cc:729] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e372f443fb0b481d9ca913ba70fb63c2, State: Initialized, Role: FOLLOWER
I20260619 20:31:52.897341 6761 consensus_queue.cc:260] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P e372f443fb0b481d9ca913ba70fb63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:52.897451 6761 ts_tablet_manager.cc:1428] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P e372f443fb0b481d9ca913ba70fb63c2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 20:31:52.969009 6741 raft_consensus.cc:482] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:31:52.969144 6741 raft_consensus.cc:504] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:52.969331 6741 leader_election.cc:288] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P e372f443fb0b481d9ca913ba70fb63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7482052ce1824e138d8b6e9b5b48e24d (127.4.83.66:40339), 5bbd2f6ea420477192319afe8301bcf4 (127.4.83.67:37647)
I20260619 20:31:52.969628 6596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dc6a0ba647a4ea5879c1e0cfb2e4847" candidate_uuid: "e372f443fb0b481d9ca913ba70fb63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bbd2f6ea420477192319afe8301bcf4" is_pre_election: true
I20260619 20:31:52.969645 6456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dc6a0ba647a4ea5879c1e0cfb2e4847" candidate_uuid: "e372f443fb0b481d9ca913ba70fb63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7482052ce1824e138d8b6e9b5b48e24d" is_pre_election: true
I20260619 20:31:52.969733 6596 raft_consensus.cc:2454] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e372f443fb0b481d9ca913ba70fb63c2 in term 0.
I20260619 20:31:52.969774 6456 raft_consensus.cc:2454] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 7482052ce1824e138d8b6e9b5b48e24d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e372f443fb0b481d9ca913ba70fb63c2 in term 0.
I20260619 20:31:52.969924 6248 leader_election.cc:302] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P e372f443fb0b481d9ca913ba70fb63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5bbd2f6ea420477192319afe8301bcf4, e372f443fb0b481d9ca913ba70fb63c2; no voters:
I20260619 20:31:52.970039 6741 raft_consensus.cc:2781] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:31:52.970081 6741 raft_consensus.cc:482] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:31:52.970103 6741 raft_consensus.cc:3037] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P e372f443fb0b481d9ca913ba70fb63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:52.971011 6741 raft_consensus.cc:504] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P e372f443fb0b481d9ca913ba70fb63c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:52.971187 6741 leader_election.cc:288] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P e372f443fb0b481d9ca913ba70fb63c2 [CANDIDATE]: Term 1 election: Requested vote from peers 7482052ce1824e138d8b6e9b5b48e24d (127.4.83.66:40339), 5bbd2f6ea420477192319afe8301bcf4 (127.4.83.67:37647)
I20260619 20:31:52.971410 6456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dc6a0ba647a4ea5879c1e0cfb2e4847" candidate_uuid: "e372f443fb0b481d9ca913ba70fb63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7482052ce1824e138d8b6e9b5b48e24d"
I20260619 20:31:52.971410 6596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dc6a0ba647a4ea5879c1e0cfb2e4847" candidate_uuid: "e372f443fb0b481d9ca913ba70fb63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bbd2f6ea420477192319afe8301bcf4"
I20260619 20:31:52.971505 6456 raft_consensus.cc:3037] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 7482052ce1824e138d8b6e9b5b48e24d [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:52.971505 6596 raft_consensus.cc:3037] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 5bbd2f6ea420477192319afe8301bcf4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:52.972479 6456 raft_consensus.cc:2454] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 7482052ce1824e138d8b6e9b5b48e24d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e372f443fb0b481d9ca913ba70fb63c2 in term 1.
I20260619 20:31:52.972479 6596 raft_consensus.cc:2454] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 5bbd2f6ea420477192319afe8301bcf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e372f443fb0b481d9ca913ba70fb63c2 in term 1.
I20260619 20:31:52.972707 6247 leader_election.cc:302] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P e372f443fb0b481d9ca913ba70fb63c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5bbd2f6ea420477192319afe8301bcf4, 7482052ce1824e138d8b6e9b5b48e24d, e372f443fb0b481d9ca913ba70fb63c2; no voters:
I20260619 20:31:52.972837 6741 raft_consensus.cc:2781] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P e372f443fb0b481d9ca913ba70fb63c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:31:52.972886 6741 raft_consensus.cc:686] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P e372f443fb0b481d9ca913ba70fb63c2 [term 1 LEADER]: Becoming Leader. State: Replica: e372f443fb0b481d9ca913ba70fb63c2, State: Running, Role: LEADER
I20260619 20:31:52.972944 6741 consensus_queue.cc:237] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P e372f443fb0b481d9ca913ba70fb63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } }
I20260619 20:31:52.973820 6159 catalog_manager.cc:5495] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P e372f443fb0b481d9ca913ba70fb63c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to e372f443fb0b481d9ca913ba70fb63c2 (127.4.83.65). New cstate: current_term: 1 leader_uuid: "e372f443fb0b481d9ca913ba70fb63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e372f443fb0b481d9ca913ba70fb63c2" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 } health_report { overall_health: UNKNOWN } } }
I20260619 20:31:52.992799 6766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:52.989850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46744 (local address 127.4.83.65:45613)
0619 20:31:52.989890 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:52.989892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:52.989901 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:31:52.989954 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:52.989955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:52.989969 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:52.990025 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:31:52.990185 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:52.990931 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:52.991942 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:52.992238 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:52.992321 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:52.992498 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:31:52.992540 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:31:52.992591 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:52.992648 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 20:31:52.992684 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:31:52.995816 6596 raft_consensus.cc:1261] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 5bbd2f6ea420477192319afe8301bcf4 [term 1 FOLLOWER]: Refusing update from remote peer e372f443fb0b481d9ca913ba70fb63c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:31:52.995945 6456 raft_consensus.cc:1261] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 7482052ce1824e138d8b6e9b5b48e24d [term 1 FOLLOWER]: Refusing update from remote peer e372f443fb0b481d9ca913ba70fb63c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:31:52.996145 6741 consensus_queue.cc:1035] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P e372f443fb0b481d9ca913ba70fb63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bbd2f6ea420477192319afe8301bcf4" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:31:52.996271 6737 consensus_queue.cc:1035] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P e372f443fb0b481d9ca913ba70fb63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7482052ce1824e138d8b6e9b5b48e24d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 20:31:53.003333 6729 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 20:31:54.526461 6159 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:31:54.526613 6159 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:31:54.527891 6159 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1164ce5a83bc48bc91a00bac7c0d9552: Sending DeleteTablet for 3 replicas of tablet 1dc6a0ba647a4ea5879c1e0cfb2e4847
I20260619 20:31:54.528244 6436 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dc6a0ba647a4ea5879c1e0cfb2e4847 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:31:54 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:50352
I20260619 20:31:54.528288 6575 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dc6a0ba647a4ea5879c1e0cfb2e4847 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:31:54 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:51444
I20260619 20:31:54.528297 6296 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dc6a0ba647a4ea5879c1e0cfb2e4847 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:31:54 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:46734
I20260619 20:31:54.528503 6811 tablet_replica.cc:330] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 5bbd2f6ea420477192319afe8301bcf4: stopping tablet replica
I20260619 20:31:54.528573 6811 raft_consensus.cc:2229] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 5bbd2f6ea420477192319afe8301bcf4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:31:54.528574 6812 tablet_replica.cc:330] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P e372f443fb0b481d9ca913ba70fb63c2: stopping tablet replica
I20260619 20:31:54.528627 6811 raft_consensus.cc:2258] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 5bbd2f6ea420477192319afe8301bcf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:31:54.528709 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 6231
I20260619 20:31:54.529481 6811 ts_tablet_manager.cc:1905] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 5bbd2f6ea420477192319afe8301bcf4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:31:54.531733 6811 ts_tablet_manager.cc:1918] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 5bbd2f6ea420477192319afe8301bcf4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 20:31:54.531817 6811 log.cc:1196] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 5bbd2f6ea420477192319afe8301bcf4: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901102012824-4429-0/minicluster-data/ts-2/wal/wals/1dc6a0ba647a4ea5879c1e0cfb2e4847
I20260619 20:31:54.532073 6811 ts_tablet_manager.cc:1939] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 5bbd2f6ea420477192319afe8301bcf4: Deleting consensus metadata
I20260619 20:31:54.532687 6144 catalog_manager.cc:4841] TS 5bbd2f6ea420477192319afe8301bcf4 (127.4.83.67:37647): tablet 1dc6a0ba647a4ea5879c1e0cfb2e4847 (table test-table [id=9e9ecaf8a76b4a9281b90964c6b3f352]) successfully deleted
I20260619 20:31:54.535879 6810 tablet_replica.cc:330] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 7482052ce1824e138d8b6e9b5b48e24d: stopping tablet replica
I20260619 20:31:54.535998 6810 raft_consensus.cc:2229] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 7482052ce1824e138d8b6e9b5b48e24d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:31:54.536069 6810 raft_consensus.cc:2258] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 7482052ce1824e138d8b6e9b5b48e24d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:31:54.536392 6810 ts_tablet_manager.cc:1905] T 1dc6a0ba647a4ea5879c1e0cfb2e4847 P 7482052ce1824e138d8b6e9b5b48e24d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 20:31:54.536880 6146 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.83.65:45613 (error 108)
I20260619 20:31:54.537089 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 6371
I20260619 20:31:54.544045 6814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:54.537763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:45613 (local address 127.0.0.1:46756)
0619 20:31:54.543894 (+ 6131us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:54.543976 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.65:45613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6053,"thread_start_us":78,"threads_started":1}
W20260619 20:31:54.544266 6146 catalog_manager.cc:4584] TS e372f443fb0b481d9ca913ba70fb63c2 (127.4.83.65:45613): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1dc6a0ba647a4ea5879c1e0cfb2e4847: Network error: Client connection negotiation failed: client connection to 127.4.83.65:45613: connect: Connection refused (error 111)
I20260619 20:31:54.544867 6814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:54.544760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:40339 (local address 127.0.0.1:50372)
0619 20:31:54.544810 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:54.544838 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.66:40339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 20:31:54.544902 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 6511
I20260619 20:31:54.550707 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 6127
2026-06-19T20:31:54Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6087 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:31:54 dist-test-slave-4m58 krb5kdc[6817](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:31:54 dist-test-slave-4m58 krb5kdc[6817](info): set up 2 sockets
Jun 19 20:31:54 dist-test-slave-4m58 krb5kdc[6817](info): commencing operation
krb5kdc: starting...
W20260619 20:31:55.552639 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.980s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:31:55 dist-test-slave-4m58 krb5kdc[6817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901115, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:31:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:31:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
I20260619 20:31:55.692777 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:33515
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:37039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:33515
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:31:55.774396 6830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:31:55.774634 6830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:31:55.774682 6830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:31:55.775944 6830 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:31:55.776001 6830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:31:55.776015 6830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:31:55.776027 6830 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:31:55.776038 6830 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:31:55.777588 6830 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:37039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:33515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:33515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:31:55.777849 6830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:31:55.778102 6830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:31:55.783517 6830 server_base.cc:1034] running on GCE node
W20260619 20:31:55.783584 6838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:55.783560 6836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:55.783517 6835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:55.784085 6830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:31:55.784437 6830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:31:55.785614 6830 hybrid_clock.cc:648] HybridClock initialized: now 1781901115785593 us; error 52 us; skew 500 ppm
Jun 19 20:31:55 dist-test-slave-4m58 krb5kdc[6817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901115, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:31:55.788600 6830 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:31:55.789165 6830 webserver.cc:460] Webserver started at http://127.4.83.126:36695/ using document root <none> and password file <none>
I20260619 20:31:55.789410 6830 fs_manager.cc:359] Metadata directory not provided
I20260619 20:31:55.789465 6830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:31:55.789597 6830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:31:55.790700 6830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "69c74880e02a44308e2fc4a738ea9d3c"
format_stamp: "Formatted at 2026-06-19 20:31:55 on dist-test-slave-4m58"
server_key: "98b03d433ad356e683c20c0a9a4b558b"
server_key_iv: "96e9ba6b3c5b981ddeef0a1ffa437bb1"
server_key_version: "encryptionkey@0"
I20260619 20:31:55.791049 6830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "69c74880e02a44308e2fc4a738ea9d3c"
format_stamp: "Formatted at 2026-06-19 20:31:55 on dist-test-slave-4m58"
server_key: "98b03d433ad356e683c20c0a9a4b558b"
server_key_iv: "96e9ba6b3c5b981ddeef0a1ffa437bb1"
server_key_version: "encryptionkey@0"
I20260619 20:31:55.792630 6830 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:31:55.793519 6845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:55.793774 6830 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:31:55.793872 6830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "69c74880e02a44308e2fc4a738ea9d3c"
format_stamp: "Formatted at 2026-06-19 20:31:55 on dist-test-slave-4m58"
server_key: "98b03d433ad356e683c20c0a9a4b558b"
server_key_iv: "96e9ba6b3c5b981ddeef0a1ffa437bb1"
server_key_version: "encryptionkey@0"
I20260619 20:31:55.793946 6830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:31:55.818192 6830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:31:55.820683 6830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:31:55.820833 6830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:31:55.825378 6830 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:33515
I20260619 20:31:55.825470 6897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:33515 every 8 connection(s)
I20260619 20:31:55.825831 6830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:31:55.826922 6898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:55.829559 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 6830
I20260619 20:31:55.829658 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:31:55.829830 4429 external_mini_cluster.cc:1428] Setting key b29a176910f97ccca9e82620b0617fa1
I20260619 20:31:55.830044 6898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69c74880e02a44308e2fc4a738ea9d3c: Bootstrap starting.
I20260619 20:31:55.830917 6898 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69c74880e02a44308e2fc4a738ea9d3c: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:55.831259 6898 log.cc:824] T 00000000000000000000000000000000 P 69c74880e02a44308e2fc4a738ea9d3c: Log is configured to *not* fsync() on all Append() calls
I20260619 20:31:55.832271 6898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69c74880e02a44308e2fc4a738ea9d3c: No bootstrap required, opened a new log
I20260619 20:31:55.834805 6898 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69c74880e02a44308e2fc4a738ea9d3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69c74880e02a44308e2fc4a738ea9d3c" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 33515 } }
I20260619 20:31:55.834986 6898 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69c74880e02a44308e2fc4a738ea9d3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:55.835021 6898 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69c74880e02a44308e2fc4a738ea9d3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69c74880e02a44308e2fc4a738ea9d3c, State: Initialized, Role: FOLLOWER
I20260619 20:31:55.835170 6898 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69c74880e02a44308e2fc4a738ea9d3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69c74880e02a44308e2fc4a738ea9d3c" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 33515 } }
I20260619 20:31:55.835261 6898 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69c74880e02a44308e2fc4a738ea9d3c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:31:55.835294 6898 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69c74880e02a44308e2fc4a738ea9d3c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:31:55.835367 6898 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69c74880e02a44308e2fc4a738ea9d3c [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:55.836476 6898 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69c74880e02a44308e2fc4a738ea9d3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69c74880e02a44308e2fc4a738ea9d3c" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 33515 } }
I20260619 20:31:55.836627 6898 leader_election.cc:302] T 00000000000000000000000000000000 P 69c74880e02a44308e2fc4a738ea9d3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69c74880e02a44308e2fc4a738ea9d3c; no voters:
I20260619 20:31:55.836908 6898 leader_election.cc:288] T 00000000000000000000000000000000 P 69c74880e02a44308e2fc4a738ea9d3c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:31:55.837036 6903 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69c74880e02a44308e2fc4a738ea9d3c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:31:55.837323 6898 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69c74880e02a44308e2fc4a738ea9d3c [sys.catalog]: configured and running, proceeding with master startup.
Jun 19 20:31:55 dist-test-slave-4m58 krb5kdc[6817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901115, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:31:55.837335 6903 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69c74880e02a44308e2fc4a738ea9d3c [term 1 LEADER]: Becoming Leader. State: Replica: 69c74880e02a44308e2fc4a738ea9d3c, State: Running, Role: LEADER
I20260619 20:31:55.839315 6903 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69c74880e02a44308e2fc4a738ea9d3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69c74880e02a44308e2fc4a738ea9d3c" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 33515 } }
I20260619 20:31:55.840329 6904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69c74880e02a44308e2fc4a738ea9d3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69c74880e02a44308e2fc4a738ea9d3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69c74880e02a44308e2fc4a738ea9d3c" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 33515 } } }
I20260619 20:31:55.840494 6904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69c74880e02a44308e2fc4a738ea9d3c [sys.catalog]: This master's current role is: LEADER
W20260619 20:31:55.841302 6919 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 69c74880e02a44308e2fc4a738ea9d3c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:31:55.841367 6919 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:31:55.841419 6918 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:31:55.840346 6905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69c74880e02a44308e2fc4a738ea9d3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69c74880e02a44308e2fc4a738ea9d3c. Latest consensus state: current_term: 1 leader_uuid: "69c74880e02a44308e2fc4a738ea9d3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69c74880e02a44308e2fc4a738ea9d3c" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 33515 } } }
I20260619 20:31:55.841540 6905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69c74880e02a44308e2fc4a738ea9d3c [sys.catalog]: This master's current role is: LEADER
I20260619 20:31:55.841653 6918 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:31:55.844024 6901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:55.830438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39710 (local address 127.4.83.126:33515)
0619 20:31:55.830624 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:55.830635 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:55.830706 (+ 71us) server_negotiation.cc:407] Connection header received
0619 20:31:55.831341 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:55.831352 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:55.831421 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:55.831598 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:55.832467 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:55.833458 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:55.834673 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:55.835192 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:55.835312 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:55.838873 (+ 3561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:55.838878 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:55.838879 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:55.838894 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:55.841346 (+ 2452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:55.842037 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:55.842042 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:55.842042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:55.842202 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:55.842504 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:55.842506 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:55.842507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:55.842871 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:55.843041 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:55.843618 (+ 577us) server_negotiation.cc:299] Negotiation successful
0619 20:31:55.843783 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":79,"threads_started":1}
I20260619 20:31:55.844179 6918 catalog_manager.cc:1347] Generated new cluster ID: fecc4d276b17456881eb44fad8bff8c9
I20260619 20:31:55.844237 6918 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:31:55.852919 6918 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:31:55.853581 6918 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:31:55.859313 6918 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69c74880e02a44308e2fc4a738ea9d3c: Generated new TSK 0
I20260619 20:31:55.859599 6918 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.83.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
I20260619 20:31:55.896946 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:0
--local_ip_for_outbound_sockets=127.4.83.65
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:33515
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:37039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:31:56.000151 6924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:31:56.000437 6924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:31:56.000480 6924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:31:56.002409 6924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:31:56.002633 6924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:31:56.002722 6924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:31:56.005069 6924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:37039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:33515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:31:56.005431 6924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:31:56.005713 6924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:31:56.011889 6932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:56.011919 6929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:56.012099 6924 server_base.cc:1034] running on GCE node
W20260619 20:31:56.011889 6930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:56.012511 6924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:31:56.012817 6924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:31:56.014022 6924 hybrid_clock.cc:648] HybridClock initialized: now 1781901116013984 us; error 56 us; skew 500 ppm
Jun 19 20:31:56 dist-test-slave-4m58 krb5kdc[6817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901116, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:31:56.017207 6924 init.cc:377] Logged in from keytab as kudu/127.4.83.65@KRBTEST.COM (short username kudu)
I20260619 20:31:56.017797 6924 webserver.cc:460] Webserver started at http://127.4.83.65:46203/ using document root <none> and password file <none>
I20260619 20:31:56.018046 6924 fs_manager.cc:359] Metadata directory not provided
I20260619 20:31:56.018105 6924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:31:56.018211 6924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:31:56.019351 6924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/data/instance:
uuid: "54b2e13938c64d5f9f9068ef48191905"
format_stamp: "Formatted at 2026-06-19 20:31:56 on dist-test-slave-4m58"
server_key: "a035154867c2e1d19e51ef46231eb912"
server_key_iv: "9d50a8b60bad9df668444d5911d85aba"
server_key_version: "encryptionkey@0"
I20260619 20:31:56.019734 6924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance:
uuid: "54b2e13938c64d5f9f9068ef48191905"
format_stamp: "Formatted at 2026-06-19 20:31:56 on dist-test-slave-4m58"
server_key: "a035154867c2e1d19e51ef46231eb912"
server_key_iv: "9d50a8b60bad9df668444d5911d85aba"
server_key_version: "encryptionkey@0"
I20260619 20:31:56.021371 6924 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:31:56.022480 6939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:56.022843 6924 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:31:56.022997 6924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "54b2e13938c64d5f9f9068ef48191905"
format_stamp: "Formatted at 2026-06-19 20:31:56 on dist-test-slave-4m58"
server_key: "a035154867c2e1d19e51ef46231eb912"
server_key_iv: "9d50a8b60bad9df668444d5911d85aba"
server_key_version: "encryptionkey@0"
I20260619 20:31:56.023079 6924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:31:56.052855 6924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:31:56.055369 6924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:31:56.055491 6924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:31:56.055759 6924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:31:56.056147 6924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:31:56.056186 6924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:56.056217 6924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:31:56.056234 6924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:56.062969 6924 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:39793
I20260619 20:31:56.063148 7052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:39793 every 8 connection(s)
I20260619 20:31:56.063410 6924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
I20260619 20:31:56.064033 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 6924
I20260619 20:31:56.064144 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance
I20260619 20:31:56.064320 4429 external_mini_cluster.cc:1428] Setting key 8a1f3f624de8cbfbb47bc56c09349338
Jun 19 20:31:56 dist-test-slave-4m58 krb5kdc[6817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901116, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.83.66@KRBTEST.COM; defaulting to no policy
I20260619 20:31:56.076596 6901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:56.064443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:48811 (local address 127.4.83.126:33515)
0619 20:31:56.064516 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:56.064519 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:56.064721 (+ 202us) server_negotiation.cc:407] Connection header received
0619 20:31:56.065269 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:56.065270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:56.065286 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:56.065367 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:56.066665 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:56.067384 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:56.068980 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:56.069253 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:56.069363 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:56.072947 (+ 3584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:56.072950 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:56.072950 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:56.072960 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:56.074532 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:56.075355 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:56.075357 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:56.075358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:56.075448 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:56.075706 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:56.075707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:56.075707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:56.075929 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:56.075995 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:56.076287 (+ 292us) server_negotiation.cc:299] Negotiation successful
0619 20:31:56.076491 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 20:31:56.077076 7055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:56.064338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:33515 (local address 127.4.83.65:48811)
0619 20:31:56.064541 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:56.064586 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:56.065147 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:56.065455 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:56.065459 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:56.066210 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:31:56.066505 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:56.066511 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:56.067547 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:56.067548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:56.068839 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:56.068843 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:56.069390 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:56.069391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:56.069563 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:56.070477 (+ 914us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:31:56.070488 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:31:56.072786 (+ 2298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:31:56.074687 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:56.074707 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:56.074711 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:56.075180 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:56.075520 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:56.075521 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:56.075521 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:56.075652 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:56.075989 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:31:56.075996 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:31:56.076156 (+ 160us) client_negotiation.cc:769] Sending connection context
0619 20:31:56.076255 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 20:31:56.076547 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":94,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.66@KRBTEST.COM" created.
I20260619 20:31:56.077849 7053 heartbeater.cc:347] Connected to a master server at 127.4.83.126:33515
I20260619 20:31:56.077982 7053 heartbeater.cc:464] Registering TS with master...
I20260619 20:31:56.078238 7053 heartbeater.cc:511] Master 127.4.83.126:33515 requested a full tablet report, sending...
I20260619 20:31:56.079047 6862 ts_manager.cc:194] Registered new tserver with Master: 54b2e13938c64d5f9f9068ef48191905 (127.4.83.65:39793)
I20260619 20:31:56.079919 6862 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.65@KRBTEST.COM'} at 127.4.83.65:48811
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
I20260619 20:31:56.097023 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:0
--local_ip_for_outbound_sockets=127.4.83.66
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:33515
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:37039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:31:56.184659 7058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:31:56.184900 7058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:31:56.184926 7058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:31:56.186160 7058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:31:56.186209 7058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:31:56.186239 7058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:31:56.187788 7058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:37039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:33515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:31:56.188081 7058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:31:56.188300 7058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:31:56.193868 7066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:56.193964 7058 server_base.cc:1034] running on GCE node
W20260619 20:31:56.193876 7064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:56.193876 7063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:56.194502 7058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:31:56.194803 7058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:31:56.195957 7058 hybrid_clock.cc:648] HybridClock initialized: now 1781901116195932 us; error 47 us; skew 500 ppm
Jun 19 20:31:56 dist-test-slave-4m58 krb5kdc[6817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901116, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:31:56.199995 7058 init.cc:377] Logged in from keytab as kudu/127.4.83.66@KRBTEST.COM (short username kudu)
I20260619 20:31:56.200603 7058 webserver.cc:460] Webserver started at http://127.4.83.66:44359/ using document root <none> and password file <none>
I20260619 20:31:56.200866 7058 fs_manager.cc:359] Metadata directory not provided
I20260619 20:31:56.200923 7058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:31:56.201071 7058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:31:56.202535 7058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/data/instance:
uuid: "71a7744720d545b48df5c8c94f633416"
format_stamp: "Formatted at 2026-06-19 20:31:56 on dist-test-slave-4m58"
server_key: "cded9b135cb88e677fba95eed71de6b7"
server_key_iv: "95513132d175ff25b0bcd3279ed38eec"
server_key_version: "encryptionkey@0"
I20260619 20:31:56.203012 7058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance:
uuid: "71a7744720d545b48df5c8c94f633416"
format_stamp: "Formatted at 2026-06-19 20:31:56 on dist-test-slave-4m58"
server_key: "cded9b135cb88e677fba95eed71de6b7"
server_key_iv: "95513132d175ff25b0bcd3279ed38eec"
server_key_version: "encryptionkey@0"
I20260619 20:31:56.204995 7058 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260619 20:31:56.206260 7073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:56.206548 7058 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:31:56.206681 7058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "71a7744720d545b48df5c8c94f633416"
format_stamp: "Formatted at 2026-06-19 20:31:56 on dist-test-slave-4m58"
server_key: "cded9b135cb88e677fba95eed71de6b7"
server_key_iv: "95513132d175ff25b0bcd3279ed38eec"
server_key_version: "encryptionkey@0"
I20260619 20:31:56.206826 7058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:31:56.225334 7058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:31:56.228372 7058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:31:56.228564 7058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:31:56.228879 7058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:31:56.229278 7058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:31:56.229315 7058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:56.229349 7058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:31:56.229372 7058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:56.238008 7058 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:46829
I20260619 20:31:56.238075 7186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:46829 every 8 connection(s)
I20260619 20:31:56.238549 7058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
I20260619 20:31:56.243619 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 7058
I20260619 20:31:56.243731 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance
I20260619 20:31:56.243973 4429 external_mini_cluster.cc:1428] Setting key e7c7b1397692a44d5590bfc4fd37cc9d
Jun 19 20:31:56 dist-test-slave-4m58 krb5kdc[6817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901116, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:31:56.248982 6901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:56.239410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:43093 (local address 127.4.83.126:33515)
0619 20:31:56.239493 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:56.239495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:56.239683 (+ 188us) server_negotiation.cc:407] Connection header received
0619 20:31:56.240263 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:56.240265 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:56.240279 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:56.240369 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:56.241408 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:56.241996 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:56.243023 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:56.243241 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:56.243313 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:56.245684 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:56.245686 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:56.245686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:56.245696 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:56.247391 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:56.247941 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:56.247942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:56.247943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:56.248017 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:56.248295 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:56.248295 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:56.248296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:56.248428 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:56.248481 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:56.248802 (+ 321us) server_negotiation.cc:299] Negotiation successful
0619 20:31:56.248893 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 20:31:56.249236 7189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:56.239344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:33515 (local address 127.4.83.66:43093)
0619 20:31:56.239546 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:56.239590 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:56.240127 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:56.240436 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:56.240440 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:56.241032 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:31:56.241238 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:56.241242 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:56.242076 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:56.242077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:56.242912 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:56.242914 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:56.243325 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:56.243325 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:56.243440 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:56.244163 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:31:56.244172 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:31:56.245586 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:31:56.247501 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:56.247507 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:56.247509 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:56.247822 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:56.248093 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:56.248093 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:56.248094 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:56.248228 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:56.248486 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:31:56.248492 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:31:56.248659 (+ 167us) client_negotiation.cc:769] Sending connection context
0619 20:31:56.248782 (+ 123us) client_negotiation.cc:240] Negotiation successful
0619 20:31:56.248983 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":116,"threads_started":1}
I20260619 20:31:56.249774 7187 heartbeater.cc:347] Connected to a master server at 127.4.83.126:33515
I20260619 20:31:56.249915 7187 heartbeater.cc:464] Registering TS with master...
I20260619 20:31:56.250162 7187 heartbeater.cc:511] Master 127.4.83.126:33515 requested a full tablet report, sending...
I20260619 20:31:56.250633 6862 ts_manager.cc:194] Registered new tserver with Master: 71a7744720d545b48df5c8c94f633416 (127.4.83.66:46829)
I20260619 20:31:56.251050 6862 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.66@KRBTEST.COM'} at 127.4.83.66:43093
WARNING: no policy specified for kudu/127.4.83.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
I20260619 20:31:56.271042 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:0
--local_ip_for_outbound_sockets=127.4.83.67
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:33515
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:37039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:31:56.377501 7192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:31:56.377736 7192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:31:56.377761 7192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:31:56.379045 7192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:31:56.379099 7192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:31:56.379129 7192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:31:56.380780 7192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:37039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:33515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:31:56.381027 7192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:31:56.381278 7192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:31:56.387094 7197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:56.387102 7198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:56.387356 7200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:56.387813 7192 server_base.cc:1034] running on GCE node
I20260619 20:31:56.388103 7192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:31:56.388389 7192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:31:56.389508 7192 hybrid_clock.cc:648] HybridClock initialized: now 1781901116389519 us; error 139 us; skew 500 ppm
Jun 19 20:31:56 dist-test-slave-4m58 krb5kdc[6817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901116, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:31:56.393889 7192 init.cc:377] Logged in from keytab as kudu/127.4.83.67@KRBTEST.COM (short username kudu)
I20260619 20:31:56.394577 7192 webserver.cc:460] Webserver started at http://127.4.83.67:44845/ using document root <none> and password file <none>
I20260619 20:31:56.394892 7192 fs_manager.cc:359] Metadata directory not provided
I20260619 20:31:56.395114 7192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:31:56.395288 7192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:31:56.397305 7192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/data/instance:
uuid: "e79871e112c947049ccdbc8570e45363"
format_stamp: "Formatted at 2026-06-19 20:31:56 on dist-test-slave-4m58"
server_key: "7d5ab330a15dc9eeab9db4d1b20ecc5d"
server_key_iv: "6b7f35485f4175266015d3621ae225a7"
server_key_version: "encryptionkey@0"
I20260619 20:31:56.397989 7192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance:
uuid: "e79871e112c947049ccdbc8570e45363"
format_stamp: "Formatted at 2026-06-19 20:31:56 on dist-test-slave-4m58"
server_key: "7d5ab330a15dc9eeab9db4d1b20ecc5d"
server_key_iv: "6b7f35485f4175266015d3621ae225a7"
server_key_version: "encryptionkey@0"
I20260619 20:31:56.400377 7192 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260619 20:31:56.401793 7207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:56.402036 7192 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:31:56.402206 7192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "e79871e112c947049ccdbc8570e45363"
format_stamp: "Formatted at 2026-06-19 20:31:56 on dist-test-slave-4m58"
server_key: "7d5ab330a15dc9eeab9db4d1b20ecc5d"
server_key_iv: "6b7f35485f4175266015d3621ae225a7"
server_key_version: "encryptionkey@0"
I20260619 20:31:56.402319 7192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:31:56.416088 7192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:31:56.419297 7192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:31:56.419487 7192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:31:56.419857 7192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:31:56.420323 7192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:31:56.420378 7192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:56.420418 7192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:31:56.420449 7192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:56.430152 7192 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:39453
I20260619 20:31:56.430214 7320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:39453 every 8 connection(s)
I20260619 20:31:56.430732 7192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
I20260619 20:31:56.438097 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 7192
I20260619 20:31:56.438222 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance
I20260619 20:31:56.438433 4429 external_mini_cluster.cc:1428] Setting key 5770991a8b77e3c481b79efb9824e677
Jun 19 20:31:56 dist-test-slave-4m58 krb5kdc[6817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901116, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:31:56.442992 6901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:56.431570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:52225 (local address 127.4.83.126:33515)
0619 20:31:56.431640 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:56.431642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:56.432004 (+ 362us) server_negotiation.cc:407] Connection header received
0619 20:31:56.432844 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:56.432846 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:56.432859 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:56.432938 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:56.434451 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:56.435261 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:56.436601 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:56.436914 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:56.436988 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:56.439938 (+ 2950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:56.439940 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:56.439940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:56.439950 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:56.441396 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:56.441954 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:56.441954 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:56.441955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:56.442014 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:56.442293 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:56.442294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:56.442294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:56.442446 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:56.442518 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:56.442837 (+ 319us) server_negotiation.cc:299] Negotiation successful
0619 20:31:56.442917 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 20:31:56.443267 7323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:56.431520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:33515 (local address 127.4.83.67:52225)
0619 20:31:56.431766 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:56.431857 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:56.432684 (+ 827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:56.433059 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:56.433065 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:56.433932 (+ 867us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:31:56.434312 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:56.434318 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:56.435397 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:56.435398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:56.436493 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:56.436496 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:56.437006 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:56.437007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:56.437158 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:56.437760 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:31:56.437773 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:31:56.439776 (+ 2003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:31:56.441522 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:56.441533 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:56.441536 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:56.441869 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:56.442075 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:56.442076 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:56.442076 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:56.442194 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:56.442513 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:31:56.442517 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:31:56.442706 (+ 189us) client_negotiation.cc:769] Sending connection context
0619 20:31:56.442809 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 20:31:56.442985 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":136,"threads_started":1}
I20260619 20:31:56.443882 7321 heartbeater.cc:347] Connected to a master server at 127.4.83.126:33515
I20260619 20:31:56.444012 7321 heartbeater.cc:464] Registering TS with master...
I20260619 20:31:56.444259 7321 heartbeater.cc:511] Master 127.4.83.126:33515 requested a full tablet report, sending...
I20260619 20:31:56.444736 6862 ts_manager.cc:194] Registered new tserver with Master: e79871e112c947049ccdbc8570e45363 (127.4.83.67:39453)
I20260619 20:31:56.445123 6862 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.67@KRBTEST.COM'} at 127.4.83.67:52225
I20260619 20:31:56.450888 4429 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 20:31:56.463789 6901 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:31:56.462673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39718 (local address 127.4.83.126:33515)
0619 20:31:56.462784 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:56.462787 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:56.462907 (+ 120us) server_negotiation.cc:407] Connection header received
0619 20:31:56.462983 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:56.462985 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:56.462998 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:56.463058 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:56.463692 (+ 634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39718: BlockingRecv error: recv got EOF from 127.0.0.1:39718 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 20:31:56.464934 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 6924
I20260619 20:31:56.469715 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 7058
I20260619 20:31:56.474897 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 7192
I20260619 20:31:56.479256 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 6830
2026-06-19T20:31:56Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1927 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:31:56 dist-test-slave-4m58 krb5kdc[7333](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:31:56 dist-test-slave-4m58 krb5kdc[7333](info): set up 2 sockets
Jun 19 20:31:56 dist-test-slave-4m58 krb5kdc[7333](info): commencing operation
krb5kdc: starting...
W20260619 20:31:57.468653 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.968s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:31:57 dist-test-slave-4m58 krb5kdc[7333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:31:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:31:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
I20260619 20:31:57.585026 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:42467
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:46211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:42467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:31:57.667229 7346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:31:57.667448 7346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:31:57.667471 7346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:31:57.668643 7346 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:31:57.668689 7346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:31:57.668701 7346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:31:57.668712 7346 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:31:57.668723 7346 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:31:57.670200 7346 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:46211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:42467
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:42467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:31:57.670396 7346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:31:57.670574 7346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:31:57.675204 7352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:57.675159 7351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:57.675207 7354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:57.675402 7346 server_base.cc:1034] running on GCE node
I20260619 20:31:57.675702 7346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:31:57.676013 7346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:31:57.677163 7346 hybrid_clock.cc:648] HybridClock initialized: now 1781901117677149 us; error 29 us; skew 500 ppm
Jun 19 20:31:57 dist-test-slave-4m58 krb5kdc[7333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901117, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:31:57.684089 7346 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:31:57.684562 7346 webserver.cc:460] Webserver started at http://127.4.83.126:37817/ using document root <none> and password file <none>
I20260619 20:31:57.684787 7346 fs_manager.cc:359] Metadata directory not provided
I20260619 20:31:57.684830 7346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:31:57.684947 7346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:31:57.686038 7346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "09b682fb2dfd412b802d11ee706850e3"
format_stamp: "Formatted at 2026-06-19 20:31:57 on dist-test-slave-4m58"
server_key: "59992432905a5a595a60e28806dc0403"
server_key_iv: "2fadf622f23f70c8534986dfbf357034"
server_key_version: "encryptionkey@0"
I20260619 20:31:57.686376 7346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "09b682fb2dfd412b802d11ee706850e3"
format_stamp: "Formatted at 2026-06-19 20:31:57 on dist-test-slave-4m58"
server_key: "59992432905a5a595a60e28806dc0403"
server_key_iv: "2fadf622f23f70c8534986dfbf357034"
server_key_version: "encryptionkey@0"
I20260619 20:31:57.687655 7346 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:31:57.688540 7361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:57.688740 7346 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:31:57.688824 7346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "09b682fb2dfd412b802d11ee706850e3"
format_stamp: "Formatted at 2026-06-19 20:31:57 on dist-test-slave-4m58"
server_key: "59992432905a5a595a60e28806dc0403"
server_key_iv: "2fadf622f23f70c8534986dfbf357034"
server_key_version: "encryptionkey@0"
I20260619 20:31:57.688895 7346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:31:57.699594 7346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:31:57.701493 7346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:31:57.701627 7346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:31:57.705391 7346 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:42467
I20260619 20:31:57.705435 7413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:42467 every 8 connection(s)
I20260619 20:31:57.705760 7346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:31:57.706646 7414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:57.709095 7414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09b682fb2dfd412b802d11ee706850e3: Bootstrap starting.
I20260619 20:31:57.709640 7414 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09b682fb2dfd412b802d11ee706850e3: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:57.709844 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 7346
I20260619 20:31:57.709931 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:31:57.709982 7414 log.cc:824] T 00000000000000000000000000000000 P 09b682fb2dfd412b802d11ee706850e3: Log is configured to *not* fsync() on all Append() calls
I20260619 20:31:57.710108 4429 external_mini_cluster.cc:1428] Setting key 73b30e18ba707073704ac8a22cf62e29
I20260619 20:31:57.710755 7414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09b682fb2dfd412b802d11ee706850e3: No bootstrap required, opened a new log
I20260619 20:31:57.712966 7414 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09b682fb2dfd412b802d11ee706850e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09b682fb2dfd412b802d11ee706850e3" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 42467 } }
I20260619 20:31:57.713115 7414 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09b682fb2dfd412b802d11ee706850e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:57.713157 7414 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09b682fb2dfd412b802d11ee706850e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09b682fb2dfd412b802d11ee706850e3, State: Initialized, Role: FOLLOWER
I20260619 20:31:57.713282 7414 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09b682fb2dfd412b802d11ee706850e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09b682fb2dfd412b802d11ee706850e3" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 42467 } }
I20260619 20:31:57.713356 7414 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09b682fb2dfd412b802d11ee706850e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:31:57.713384 7414 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09b682fb2dfd412b802d11ee706850e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:31:57.713429 7414 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09b682fb2dfd412b802d11ee706850e3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:57.713965 7414 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09b682fb2dfd412b802d11ee706850e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09b682fb2dfd412b802d11ee706850e3" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 42467 } }
I20260619 20:31:57.714059 7414 leader_election.cc:302] T 00000000000000000000000000000000 P 09b682fb2dfd412b802d11ee706850e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09b682fb2dfd412b802d11ee706850e3; no voters:
I20260619 20:31:57.714226 7414 leader_election.cc:288] T 00000000000000000000000000000000 P 09b682fb2dfd412b802d11ee706850e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:31:57.714274 7419 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09b682fb2dfd412b802d11ee706850e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:31:57.714427 7419 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09b682fb2dfd412b802d11ee706850e3 [term 1 LEADER]: Becoming Leader. State: Replica: 09b682fb2dfd412b802d11ee706850e3, State: Running, Role: LEADER
I20260619 20:31:57.714465 7414 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09b682fb2dfd412b802d11ee706850e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:31:57.714510 7419 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09b682fb2dfd412b802d11ee706850e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09b682fb2dfd412b802d11ee706850e3" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 42467 } }
I20260619 20:31:57.715131 7421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09b682fb2dfd412b802d11ee706850e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09b682fb2dfd412b802d11ee706850e3. Latest consensus state: current_term: 1 leader_uuid: "09b682fb2dfd412b802d11ee706850e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09b682fb2dfd412b802d11ee706850e3" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 42467 } } }
I20260619 20:31:57.715145 7420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09b682fb2dfd412b802d11ee706850e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09b682fb2dfd412b802d11ee706850e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09b682fb2dfd412b802d11ee706850e3" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 42467 } } }
I20260619 20:31:57.715230 7421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09b682fb2dfd412b802d11ee706850e3 [sys.catalog]: This master's current role is: LEADER
I20260619 20:31:57.715302 7420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09b682fb2dfd412b802d11ee706850e3 [sys.catalog]: This master's current role is: LEADER
W20260619 20:31:57.716289 7434 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 09b682fb2dfd412b802d11ee706850e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:31:57.717392 7434 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:31:57.717458 7428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:31:57.717650 7428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 20:31:57 dist-test-slave-4m58 krb5kdc[7333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:31:57.719408 7428 catalog_manager.cc:1347] Generated new cluster ID: 3b18a5d6706d43fc9424e0af471d1b87
I20260619 20:31:57.719463 7428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:31:57.721561 7418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:57.710595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39570 (local address 127.4.83.126:42467)
0619 20:31:57.710805 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:57.710812 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:57.710843 (+ 31us) server_negotiation.cc:407] Connection header received
0619 20:31:57.711303 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:57.711309 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:57.711356 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:57.711476 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:57.712150 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:57.712996 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:57.713852 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:57.714331 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:57.714472 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:57.718288 (+ 3816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:57.718292 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:57.718293 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:57.718304 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:57.720213 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:57.720531 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:57.720534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:57.720534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:57.720618 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:57.720787 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:57.720787 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:57.720788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:57.721025 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:57.721188 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:57.721259 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 20:31:57.721389 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":126,"threads_started":1}
I20260619 20:31:57.722442 4429 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:31:57.726802 7428 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:31:57.727360 7428 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:31:57.731946 7428 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09b682fb2dfd412b802d11ee706850e3: Generated new TSK 0
I20260619 20:31:57.732136 7428 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:31:57.748237 7418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:57.744321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39576 (local address 127.4.83.126:42467)
0619 20:31:57.744375 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:57.744377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:57.744455 (+ 78us) server_negotiation.cc:407] Connection header received
0619 20:31:57.744551 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:57.744552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:57.744563 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:57.744621 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:57.745075 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:57.745661 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:57.746739 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:57.746960 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:57.747014 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:57.747708 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:57.747709 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:57.747709 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:31:57.747955 (+ 246us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:57.747974 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:31:57.747980 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:57.748034 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:57.748094 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 20:31:57.748121 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 20:31:57.750851 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 7346
2026-06-19T20:31:57Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1268 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:31:57 dist-test-slave-4m58 krb5kdc[7447](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:31:57 dist-test-slave-4m58 krb5kdc[7447](info): set up 2 sockets
Jun 19 20:31:57 dist-test-slave-4m58 krb5kdc[7447](info): commencing operation
krb5kdc: starting...
W20260619 20:31:58.716539 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.948s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:31:58 dist-test-slave-4m58 krb5kdc[7447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:31:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:31:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
I20260619 20:31:58.832543 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:44825
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:34335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:44825
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:31:58.912499 7460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:31:58.912735 7460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:31:58.912768 7460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:31:58.913962 7460 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:31:58.914007 7460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:31:58.914026 7460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:31:58.914042 7460 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:31:58.914058 7460 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:31:58.915688 7460 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:34335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:44825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:44825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:31:58.915982 7460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:31:58.916201 7460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:31:58.921329 7465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:58.921307 7466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:58.921307 7468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:58.921489 7460 server_base.cc:1034] running on GCE node
I20260619 20:31:58.921813 7460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:31:58.922135 7460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:31:58.923381 7460 hybrid_clock.cc:648] HybridClock initialized: now 1781901118923314 us; error 38 us; skew 500 ppm
Jun 19 20:31:58 dist-test-slave-4m58 krb5kdc[7447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901118, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:31:58.926120 7460 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:31:58.926637 7460 webserver.cc:460] Webserver started at http://127.4.83.126:45209/ using document root <none> and password file <none>
I20260619 20:31:58.926865 7460 fs_manager.cc:359] Metadata directory not provided
I20260619 20:31:58.926914 7460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:31:58.927047 7460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:31:58.928218 7460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "3fc97b3ea88c4acd92a9307f239bbb34"
format_stamp: "Formatted at 2026-06-19 20:31:58 on dist-test-slave-4m58"
server_key: "6c79de6c4313f6ba1bba42340e3a94f5"
server_key_iv: "e042c21ef43ac7d114537027bb79b9a8"
server_key_version: "encryptionkey@0"
I20260619 20:31:58.928555 7460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "3fc97b3ea88c4acd92a9307f239bbb34"
format_stamp: "Formatted at 2026-06-19 20:31:58 on dist-test-slave-4m58"
server_key: "6c79de6c4313f6ba1bba42340e3a94f5"
server_key_iv: "e042c21ef43ac7d114537027bb79b9a8"
server_key_version: "encryptionkey@0"
I20260619 20:31:58.929960 7460 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:31:58.930783 7475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:58.930997 7460 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:31:58.931088 7460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "3fc97b3ea88c4acd92a9307f239bbb34"
format_stamp: "Formatted at 2026-06-19 20:31:58 on dist-test-slave-4m58"
server_key: "6c79de6c4313f6ba1bba42340e3a94f5"
server_key_iv: "e042c21ef43ac7d114537027bb79b9a8"
server_key_version: "encryptionkey@0"
I20260619 20:31:58.931154 7460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:31:58.947824 7460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:31:58.949815 7460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:31:58.949955 7460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:31:58.953819 7460 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:44825
I20260619 20:31:58.953859 7527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:44825 every 8 connection(s)
I20260619 20:31:58.954192 7460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:31:58.955149 7528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:31:58.957279 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 7460
I20260619 20:31:58.957381 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:31:58.957476 7528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fc97b3ea88c4acd92a9307f239bbb34: Bootstrap starting.
I20260619 20:31:58.957561 4429 external_mini_cluster.cc:1428] Setting key 4653f4466939dc903190681e2410bedf
I20260619 20:31:58.958145 7528 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fc97b3ea88c4acd92a9307f239bbb34: Neither blocks nor log segments found. Creating new log.
I20260619 20:31:58.958420 7528 log.cc:824] T 00000000000000000000000000000000 P 3fc97b3ea88c4acd92a9307f239bbb34: Log is configured to *not* fsync() on all Append() calls
I20260619 20:31:58.959170 7528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fc97b3ea88c4acd92a9307f239bbb34: No bootstrap required, opened a new log
I20260619 20:31:58.961326 7528 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fc97b3ea88c4acd92a9307f239bbb34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fc97b3ea88c4acd92a9307f239bbb34" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 44825 } }
I20260619 20:31:58.961513 7528 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3fc97b3ea88c4acd92a9307f239bbb34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:31:58.961544 7528 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fc97b3ea88c4acd92a9307f239bbb34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fc97b3ea88c4acd92a9307f239bbb34, State: Initialized, Role: FOLLOWER
I20260619 20:31:58.961712 7528 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fc97b3ea88c4acd92a9307f239bbb34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fc97b3ea88c4acd92a9307f239bbb34" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 44825 } }
I20260619 20:31:58.961813 7528 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3fc97b3ea88c4acd92a9307f239bbb34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:31:58.961839 7528 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3fc97b3ea88c4acd92a9307f239bbb34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:31:58.961887 7528 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fc97b3ea88c4acd92a9307f239bbb34 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:31:58.962555 7528 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3fc97b3ea88c4acd92a9307f239bbb34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fc97b3ea88c4acd92a9307f239bbb34" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 44825 } }
I20260619 20:31:58.962663 7528 leader_election.cc:302] T 00000000000000000000000000000000 P 3fc97b3ea88c4acd92a9307f239bbb34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3fc97b3ea88c4acd92a9307f239bbb34; no voters:
I20260619 20:31:58.962828 7528 leader_election.cc:288] T 00000000000000000000000000000000 P 3fc97b3ea88c4acd92a9307f239bbb34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:31:58.962967 7533 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3fc97b3ea88c4acd92a9307f239bbb34 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:31:58.963099 7528 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fc97b3ea88c4acd92a9307f239bbb34 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:31:58.963160 7533 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3fc97b3ea88c4acd92a9307f239bbb34 [term 1 LEADER]: Becoming Leader. State: Replica: 3fc97b3ea88c4acd92a9307f239bbb34, State: Running, Role: LEADER
I20260619 20:31:58.963284 7533 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3fc97b3ea88c4acd92a9307f239bbb34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fc97b3ea88c4acd92a9307f239bbb34" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 44825 } }
I20260619 20:31:58.963693 7535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fc97b3ea88c4acd92a9307f239bbb34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fc97b3ea88c4acd92a9307f239bbb34. Latest consensus state: current_term: 1 leader_uuid: "3fc97b3ea88c4acd92a9307f239bbb34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fc97b3ea88c4acd92a9307f239bbb34" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 44825 } } }
I20260619 20:31:58.963698 7534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fc97b3ea88c4acd92a9307f239bbb34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3fc97b3ea88c4acd92a9307f239bbb34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fc97b3ea88c4acd92a9307f239bbb34" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 44825 } } }
I20260619 20:31:58.963788 7535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fc97b3ea88c4acd92a9307f239bbb34 [sys.catalog]: This master's current role is: LEADER
I20260619 20:31:58.963791 7534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fc97b3ea88c4acd92a9307f239bbb34 [sys.catalog]: This master's current role is: LEADER
I20260619 20:31:58.964171 7542 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Jun 19 20:31:58 dist-test-slave-4m58 krb5kdc[7447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:31:58.964679 7542 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:31:58.966833 7542 catalog_manager.cc:1347] Generated new cluster ID: 2a5bc24b7658490884941b084b047871
I20260619 20:31:58.966883 7542 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:31:58.969494 7531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:58.958058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50212 (local address 127.4.83.126:44825)
0619 20:31:58.958219 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:58.958229 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:58.958260 (+ 31us) server_negotiation.cc:407] Connection header received
0619 20:31:58.958829 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:58.958839 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:58.958892 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:58.959010 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:58.959809 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:58.960719 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:58.961758 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:58.962228 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:58.962315 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:58.965227 (+ 2912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:58.965231 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:58.965232 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:58.965245 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:58.967322 (+ 2077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:58.967760 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:58.967763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:58.967763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:58.967906 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:58.968122 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:58.968123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:58.968123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:58.968470 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:58.968622 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:58.969093 (+ 471us) server_negotiation.cc:299] Negotiation successful
0619 20:31:58.969256 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
I20260619 20:31:58.972636 7542 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:31:58.973197 7542 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:31:58.977769 7542 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3fc97b3ea88c4acd92a9307f239bbb34: Generated new TSK 0
I20260619 20:31:58.977972 7542 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.83.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
I20260619 20:31:59.003085 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:0
--local_ip_for_outbound_sockets=127.4.83.65
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:44825
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:34335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:31:59.084257 7554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:31:59.084504 7554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:31:59.084540 7554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:31:59.085831 7554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:31:59.085886 7554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:31:59.085927 7554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:31:59.087688 7554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:34335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:44825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:31:59.088009 7554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:31:59.088249 7554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:31:59.093516 7559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:59.093534 7560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:59.093528 7554 server_base.cc:1034] running on GCE node
W20260619 20:31:59.093541 7562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:59.093894 7554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:31:59.094157 7554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:31:59.095345 7554 hybrid_clock.cc:648] HybridClock initialized: now 1781901119095327 us; error 28 us; skew 500 ppm
Jun 19 20:31:59 dist-test-slave-4m58 krb5kdc[7447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901119, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:31:59.097934 7554 init.cc:377] Logged in from keytab as kudu/127.4.83.65@KRBTEST.COM (short username kudu)
I20260619 20:31:59.098416 7554 webserver.cc:460] Webserver started at http://127.4.83.65:42843/ using document root <none> and password file <none>
I20260619 20:31:59.098623 7554 fs_manager.cc:359] Metadata directory not provided
I20260619 20:31:59.098672 7554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:31:59.098800 7554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:31:59.099905 7554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/data/instance:
uuid: "651914c926464097909fa5b2228cdfc4"
format_stamp: "Formatted at 2026-06-19 20:31:59 on dist-test-slave-4m58"
server_key: "9d08b02679deb260d5178832acceeaad"
server_key_iv: "c7fad545dc9fd6c111f7eb18093ae805"
server_key_version: "encryptionkey@0"
I20260619 20:31:59.100260 7554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance:
uuid: "651914c926464097909fa5b2228cdfc4"
format_stamp: "Formatted at 2026-06-19 20:31:59 on dist-test-slave-4m58"
server_key: "9d08b02679deb260d5178832acceeaad"
server_key_iv: "c7fad545dc9fd6c111f7eb18093ae805"
server_key_version: "encryptionkey@0"
I20260619 20:31:59.101622 7554 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:31:59.102416 7569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:59.102602 7554 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:31:59.102695 7554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "651914c926464097909fa5b2228cdfc4"
format_stamp: "Formatted at 2026-06-19 20:31:59 on dist-test-slave-4m58"
server_key: "9d08b02679deb260d5178832acceeaad"
server_key_iv: "c7fad545dc9fd6c111f7eb18093ae805"
server_key_version: "encryptionkey@0"
I20260619 20:31:59.102762 7554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:31:59.126514 7554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:31:59.129038 7554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:31:59.129230 7554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:31:59.129468 7554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:31:59.129779 7554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:31:59.129813 7554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:59.129844 7554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:31:59.129861 7554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:59.135610 7554 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:44833
I20260619 20:31:59.135659 7682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:44833 every 8 connection(s)
I20260619 20:31:59.136022 7554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
I20260619 20:31:59.138494 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 7554
I20260619 20:31:59.138593 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance
I20260619 20:31:59.138818 4429 external_mini_cluster.cc:1428] Setting key b7229a0c53f4984aff3da21886e4c087
Jun 19 20:31:59 dist-test-slave-4m58 krb5kdc[7447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901119, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.83.66@KRBTEST.COM; defaulting to no policy
I20260619 20:31:59.146150 7531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:59.136812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:42613 (local address 127.4.83.126:44825)
0619 20:31:59.136886 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:59.136889 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:59.137049 (+ 160us) server_negotiation.cc:407] Connection header received
0619 20:31:59.137573 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:59.137574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:59.137587 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:59.137635 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:59.138901 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:59.139641 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:59.140714 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:59.140947 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:59.141005 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:59.143358 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:59.143360 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:59.143360 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:59.143368 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:59.144828 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:59.145224 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:59.145225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:59.145225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:59.145286 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:59.145502 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:59.145502 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:59.145503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:59.145618 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:59.145683 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:59.145945 (+ 262us) server_negotiation.cc:299] Negotiation successful
0619 20:31:59.146051 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 20:31:59.146317 7685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:59.136736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:44825 (local address 127.4.83.65:42613)
0619 20:31:59.136926 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:59.136968 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:59.137463 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:59.137708 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:59.137712 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:59.138419 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:31:59.138787 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:59.138795 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:59.139737 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:59.139738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:59.140624 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:59.140627 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:59.141026 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:59.141027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:59.141159 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:59.141696 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:31:59.141706 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:31:59.143263 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:31:59.144909 (+ 1646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:59.144917 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:59.144920 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:59.145142 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:59.145341 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:59.145341 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:59.145342 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:59.145448 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:59.145673 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:31:59.145679 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:31:59.145828 (+ 149us) client_negotiation.cc:769] Sending connection context
0619 20:31:59.145918 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 20:31:59.146083 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":92,"threads_started":1}
I20260619 20:31:59.146781 7683 heartbeater.cc:347] Connected to a master server at 127.4.83.126:44825
I20260619 20:31:59.146880 7683 heartbeater.cc:464] Registering TS with master...
I20260619 20:31:59.147096 7683 heartbeater.cc:511] Master 127.4.83.126:44825 requested a full tablet report, sending...
I20260619 20:31:59.147588 7492 ts_manager.cc:194] Registered new tserver with Master: 651914c926464097909fa5b2228cdfc4 (127.4.83.65:44833)
I20260619 20:31:59.148288 7492 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.65@KRBTEST.COM'} at 127.4.83.65:42613
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
I20260619 20:31:59.164012 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:0
--local_ip_for_outbound_sockets=127.4.83.66
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:44825
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:34335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:31:59.247722 7688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:31:59.248013 7688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:31:59.248054 7688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:31:59.249325 7688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:31:59.249383 7688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:31:59.249423 7688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:31:59.251102 7688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:34335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:44825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:31:59.251402 7688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:31:59.251641 7688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:31:59.256734 7693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:59.256865 7696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:59.256736 7694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:59.256968 7688 server_base.cc:1034] running on GCE node
I20260619 20:31:59.257164 7688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:31:59.257409 7688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:31:59.258562 7688 hybrid_clock.cc:648] HybridClock initialized: now 1781901119258547 us; error 32 us; skew 500 ppm
Jun 19 20:31:59 dist-test-slave-4m58 krb5kdc[7447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901119, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:31:59.261448 7688 init.cc:377] Logged in from keytab as kudu/127.4.83.66@KRBTEST.COM (short username kudu)
I20260619 20:31:59.261970 7688 webserver.cc:460] Webserver started at http://127.4.83.66:41113/ using document root <none> and password file <none>
I20260619 20:31:59.262180 7688 fs_manager.cc:359] Metadata directory not provided
I20260619 20:31:59.262229 7688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:31:59.262356 7688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:31:59.263514 7688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/data/instance:
uuid: "4592bd332b434e589b5003bf0ead5fbe"
format_stamp: "Formatted at 2026-06-19 20:31:59 on dist-test-slave-4m58"
server_key: "2e8554d8d31aa6be8a372a2c7cee06b6"
server_key_iv: "26595a61c46339f602a7d7c9643a195d"
server_key_version: "encryptionkey@0"
I20260619 20:31:59.263896 7688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance:
uuid: "4592bd332b434e589b5003bf0ead5fbe"
format_stamp: "Formatted at 2026-06-19 20:31:59 on dist-test-slave-4m58"
server_key: "2e8554d8d31aa6be8a372a2c7cee06b6"
server_key_iv: "26595a61c46339f602a7d7c9643a195d"
server_key_version: "encryptionkey@0"
I20260619 20:31:59.265314 7688 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:31:59.266104 7703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:59.266268 7688 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:31:59.266356 7688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "4592bd332b434e589b5003bf0ead5fbe"
format_stamp: "Formatted at 2026-06-19 20:31:59 on dist-test-slave-4m58"
server_key: "2e8554d8d31aa6be8a372a2c7cee06b6"
server_key_iv: "26595a61c46339f602a7d7c9643a195d"
server_key_version: "encryptionkey@0"
I20260619 20:31:59.266428 7688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:31:59.284739 7688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:31:59.286989 7688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:31:59.287140 7688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:31:59.287374 7688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:31:59.287691 7688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:31:59.287726 7688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:59.287755 7688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:31:59.287789 7688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:59.293879 7688 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:46485
I20260619 20:31:59.293924 7816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:46485 every 8 connection(s)
I20260619 20:31:59.294269 7688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
I20260619 20:31:59.299676 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 7688
I20260619 20:31:59.299784 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance
I20260619 20:31:59.300016 4429 external_mini_cluster.cc:1428] Setting key 04af7ef2f9308c94a01d000656c42c9c
Jun 19 20:31:59 dist-test-slave-4m58 krb5kdc[7447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901119, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:31:59.304800 7531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:59.295226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:58095 (local address 127.4.83.126:44825)
0619 20:31:59.295378 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:59.295381 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:59.295541 (+ 160us) server_negotiation.cc:407] Connection header received
0619 20:31:59.296128 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:59.296130 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:59.296141 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:59.296190 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:59.297173 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:59.297681 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:59.298914 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:59.299119 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:59.299176 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:59.301620 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:59.301622 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:59.301622 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:59.301631 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:59.303012 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:59.303533 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:59.303534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:59.303534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:59.303599 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:59.303963 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:59.303964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:59.303964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:59.304116 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:59.304189 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:59.304597 (+ 408us) server_negotiation.cc:299] Negotiation successful
0619 20:31:59.304695 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260619 20:31:59.304948 7819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:59.295167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:44825 (local address 127.4.83.66:58095)
0619 20:31:59.295397 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:59.295458 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:59.296034 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:59.296293 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:59.296296 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:59.296877 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:31:59.297076 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:59.297080 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:59.297805 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:59.297806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:59.298814 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:59.298817 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:59.299191 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:59.299192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:59.299333 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:59.299920 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:31:59.299928 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:31:59.301529 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:31:59.303166 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:59.303174 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:59.303176 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:59.303456 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:59.303718 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:59.303719 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:59.303719 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:59.303890 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:59.304207 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:31:59.304215 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:31:59.304406 (+ 191us) client_negotiation.cc:769] Sending connection context
0619 20:31:59.304525 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 20:31:59.304691 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":95,"threads_started":1}
I20260619 20:31:59.305564 7817 heartbeater.cc:347] Connected to a master server at 127.4.83.126:44825
I20260619 20:31:59.305680 7817 heartbeater.cc:464] Registering TS with master...
I20260619 20:31:59.305850 7817 heartbeater.cc:511] Master 127.4.83.126:44825 requested a full tablet report, sending...
I20260619 20:31:59.306327 7492 ts_manager.cc:194] Registered new tserver with Master: 4592bd332b434e589b5003bf0ead5fbe (127.4.83.66:46485)
I20260619 20:31:59.306877 7492 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.66@KRBTEST.COM'} at 127.4.83.66:58095
WARNING: no policy specified for kudu/127.4.83.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
I20260619 20:31:59.328066 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:0
--local_ip_for_outbound_sockets=127.4.83.67
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:44825
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:34335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:31:59.412848 7822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:31:59.413064 7822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:31:59.413089 7822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:31:59.414255 7822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:31:59.414296 7822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:31:59.414322 7822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:31:59.415771 7822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:34335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:44825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:31:59.415992 7822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:31:59.416188 7822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:31:59.421422 7830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:59.421593 7822 server_base.cc:1034] running on GCE node
W20260619 20:31:59.421424 7828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:31:59.421422 7827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:31:59.421922 7822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:31:59.422156 7822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:31:59.423314 7822 hybrid_clock.cc:648] HybridClock initialized: now 1781901119423294 us; error 41 us; skew 500 ppm
Jun 19 20:31:59 dist-test-slave-4m58 krb5kdc[7447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901119, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:31:59.425843 7822 init.cc:377] Logged in from keytab as kudu/127.4.83.67@KRBTEST.COM (short username kudu)
I20260619 20:31:59.426286 7822 webserver.cc:460] Webserver started at http://127.4.83.67:40641/ using document root <none> and password file <none>
I20260619 20:31:59.426484 7822 fs_manager.cc:359] Metadata directory not provided
I20260619 20:31:59.426530 7822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:31:59.426656 7822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:31:59.427699 7822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/data/instance:
uuid: "70f7d4b29ed4415b9b57a57accf22c4a"
format_stamp: "Formatted at 2026-06-19 20:31:59 on dist-test-slave-4m58"
server_key: "e16915de5858f2c3ca76f1c4c7639855"
server_key_iv: "c862d8f33795a5a95c6beca974f94d7d"
server_key_version: "encryptionkey@0"
I20260619 20:31:59.428072 7822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance:
uuid: "70f7d4b29ed4415b9b57a57accf22c4a"
format_stamp: "Formatted at 2026-06-19 20:31:59 on dist-test-slave-4m58"
server_key: "e16915de5858f2c3ca76f1c4c7639855"
server_key_iv: "c862d8f33795a5a95c6beca974f94d7d"
server_key_version: "encryptionkey@0"
I20260619 20:31:59.429306 7822 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:31:59.429929 7837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:59.430091 7822 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:59.430172 7822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "70f7d4b29ed4415b9b57a57accf22c4a"
format_stamp: "Formatted at 2026-06-19 20:31:59 on dist-test-slave-4m58"
server_key: "e16915de5858f2c3ca76f1c4c7639855"
server_key_iv: "c862d8f33795a5a95c6beca974f94d7d"
server_key_version: "encryptionkey@0"
I20260619 20:31:59.430234 7822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:31:59.451745 7822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:31:59.454003 7822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:31:59.454147 7822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:31:59.454365 7822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:31:59.454686 7822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:31:59.454720 7822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:59.454749 7822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:31:59.454780 7822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:31:59.460667 7822 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:35759
I20260619 20:31:59.460707 7950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:35759 every 8 connection(s)
I20260619 20:31:59.461019 7822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
I20260619 20:31:59.463039 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 7822
I20260619 20:31:59.463140 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance
I20260619 20:31:59.463308 4429 external_mini_cluster.cc:1428] Setting key cb433ff47272d8e9e05cdbeeed49b27f
Jun 19 20:31:59 dist-test-slave-4m58 krb5kdc[7447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901119, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:31:59.471202 7531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:59.461692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:53723 (local address 127.4.83.126:44825)
0619 20:31:59.461774 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:59.461776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:59.461922 (+ 146us) server_negotiation.cc:407] Connection header received
0619 20:31:59.462398 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:59.462399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:59.462408 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:59.462453 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:59.463485 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:59.464241 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:59.465342 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:59.465538 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:59.465593 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:59.468091 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:59.468093 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:59.468093 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:59.468101 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:59.469561 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:59.470047 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:59.470048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:59.470049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:59.470146 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:59.470487 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:59.470488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:59.470489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:59.470631 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:59.470697 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:59.471015 (+ 318us) server_negotiation.cc:299] Negotiation successful
0619 20:31:59.471108 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 20:31:59.471388 7953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:59.461651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:44825 (local address 127.4.83.67:53723)
0619 20:31:59.461806 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 20:31:59.461845 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 20:31:59.462327 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:31:59.462514 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:31:59.462518 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:31:59.463142 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:31:59.463367 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:59.463373 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:59.464329 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:59.464330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:59.465272 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:31:59.465274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:31:59.465615 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:31:59.465616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:31:59.465719 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:59.466295 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:31:59.466303 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:31:59.467967 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:31:59.469660 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:59.469668 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:59.469670 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:59.469953 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:59.470236 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:31:59.470237 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:31:59.470238 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:31:59.470363 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:31:59.470695 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:31:59.470703 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:31:59.470875 (+ 172us) client_negotiation.cc:769] Sending connection context
0619 20:31:59.470981 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 20:31:59.471154 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":67,"threads_started":1}
I20260619 20:31:59.471884 7951 heartbeater.cc:347] Connected to a master server at 127.4.83.126:44825
I20260619 20:31:59.471979 7951 heartbeater.cc:464] Registering TS with master...
I20260619 20:31:59.472150 7951 heartbeater.cc:511] Master 127.4.83.126:44825 requested a full tablet report, sending...
I20260619 20:31:59.472535 7492 ts_manager.cc:194] Registered new tserver with Master: 70f7d4b29ed4415b9b57a57accf22c4a (127.4.83.67:35759)
I20260619 20:31:59.472945 7492 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.67@KRBTEST.COM'} at 127.4.83.67:53723
I20260619 20:31:59.474967 4429 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 20:31:59 dist-test-slave-4m58 krb5kdc[7447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901119, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 19 20:31:59 dist-test-slave-4m58 krb5kdc[7447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901119, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:31:59.498278 7531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:31:59.490382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50222 (local address 127.4.83.126:44825)
0619 20:31:59.490459 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 20:31:59.490460 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:31:59.490540 (+ 80us) server_negotiation.cc:407] Connection header received
0619 20:31:59.490574 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:31:59.490575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:31:59.490586 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:31:59.490651 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:31:59.491284 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:59.491809 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:59.492644 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:31:59.492849 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:31:59.492909 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:31:59.495338 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:31:59.495341 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:31:59.495341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:31:59.495350 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:31:59.496756 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:59.497347 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:59.497349 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:59.497349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:59.497413 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:31:59.497678 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:31:59.497679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:31:59.497679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:31:59.497904 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:31:59.498002 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 20:31:59.498127 (+ 125us) server_negotiation.cc:299] Negotiation successful
0619 20:31:59.498202 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260619 20:31:59.498536 7492 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:50222
I20260619 20:31:59.499223 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 7554
I20260619 20:31:59.503496 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 7688
I20260619 20:31:59.507606 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 7822
I20260619 20:31:59.511390 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 7460
2026-06-19T20:31:59Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1762 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:31:59 dist-test-slave-4m58 krb5kdc[7963](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:31:59 dist-test-slave-4m58 krb5kdc[7963](info): set up 2 sockets
Jun 19 20:31:59 dist-test-slave-4m58 krb5kdc[7963](info): commencing operation
krb5kdc: starting...
W20260619 20:32:00.468530 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.938s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:32:00 dist-test-slave-4m58 krb5kdc[7963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:32:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:32:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
I20260619 20:32:00.588466 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:33041
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:43873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:33041
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:00.667896 7976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:00.668105 7976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:00.668128 7976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:00.669275 7976 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:32:00.669322 7976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:00.669334 7976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:00.669345 7976 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:32:00.669356 7976 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:32:00.670921 7976 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:43873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:33041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:33041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:00.671159 7976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:00.671371 7976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:00.676158 7982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:00.676160 7984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:00.676162 7981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:00.676393 7976 server_base.cc:1034] running on GCE node
I20260619 20:32:00.676630 7976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:00.676923 7976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:00.678094 7976 hybrid_clock.cc:648] HybridClock initialized: now 1781901120678046 us; error 66 us; skew 500 ppm
Jun 19 20:32:00 dist-test-slave-4m58 krb5kdc[7963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901120, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:00.680603 7976 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:32:00.680994 7976 webserver.cc:460] Webserver started at http://127.4.83.126:46605/ using document root <none> and password file <none>
I20260619 20:32:00.681231 7976 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:00.681290 7976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:00.681404 7976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:00.682456 7976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "98c29b9ddada432fb88d5bb8e561b2e2"
format_stamp: "Formatted at 2026-06-19 20:32:00 on dist-test-slave-4m58"
server_key: "39c84ac9120c5fc07c413a49e0ed65a2"
server_key_iv: "9b2793f4cba2e98b1990d1357648f597"
server_key_version: "encryptionkey@0"
I20260619 20:32:00.682776 7976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "98c29b9ddada432fb88d5bb8e561b2e2"
format_stamp: "Formatted at 2026-06-19 20:32:00 on dist-test-slave-4m58"
server_key: "39c84ac9120c5fc07c413a49e0ed65a2"
server_key_iv: "9b2793f4cba2e98b1990d1357648f597"
server_key_version: "encryptionkey@0"
I20260619 20:32:00.684056 7976 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:32:00.684788 7991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:00.684926 7976 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:00.685001 7976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "98c29b9ddada432fb88d5bb8e561b2e2"
format_stamp: "Formatted at 2026-06-19 20:32:00 on dist-test-slave-4m58"
server_key: "39c84ac9120c5fc07c413a49e0ed65a2"
server_key_iv: "9b2793f4cba2e98b1990d1357648f597"
server_key_version: "encryptionkey@0"
I20260619 20:32:00.685067 7976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:00.701524 7976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:00.703392 7976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:00.703500 7976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:00.707015 7976 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:33041
I20260619 20:32:00.707054 8043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:33041 every 8 connection(s)
I20260619 20:32:00.707487 7976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:32:00.708158 8044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:00.710340 8044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98c29b9ddada432fb88d5bb8e561b2e2: Bootstrap starting.
I20260619 20:32:00.710891 8044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98c29b9ddada432fb88d5bb8e561b2e2: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:00.711148 8044 log.cc:824] T 00000000000000000000000000000000 P 98c29b9ddada432fb88d5bb8e561b2e2: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:00.711853 8044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98c29b9ddada432fb88d5bb8e561b2e2: No bootstrap required, opened a new log
I20260619 20:32:00.713220 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 7976
I20260619 20:32:00.713305 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:32:00.713464 4429 external_mini_cluster.cc:1428] Setting key 13e260e3382675ea566b1063cac74f88
I20260619 20:32:00.713410 8044 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98c29b9ddada432fb88d5bb8e561b2e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98c29b9ddada432fb88d5bb8e561b2e2" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 33041 } }
I20260619 20:32:00.713559 8044 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98c29b9ddada432fb88d5bb8e561b2e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:00.713598 8044 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98c29b9ddada432fb88d5bb8e561b2e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98c29b9ddada432fb88d5bb8e561b2e2, State: Initialized, Role: FOLLOWER
I20260619 20:32:00.714015 8044 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98c29b9ddada432fb88d5bb8e561b2e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98c29b9ddada432fb88d5bb8e561b2e2" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 33041 } }
I20260619 20:32:00.714093 8044 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98c29b9ddada432fb88d5bb8e561b2e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:32:00.714123 8044 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98c29b9ddada432fb88d5bb8e561b2e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:32:00.714169 8044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98c29b9ddada432fb88d5bb8e561b2e2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:00.714682 8044 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98c29b9ddada432fb88d5bb8e561b2e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98c29b9ddada432fb88d5bb8e561b2e2" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 33041 } }
I20260619 20:32:00.714784 8044 leader_election.cc:302] T 00000000000000000000000000000000 P 98c29b9ddada432fb88d5bb8e561b2e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98c29b9ddada432fb88d5bb8e561b2e2; no voters:
I20260619 20:32:00.714943 8044 leader_election.cc:288] T 00000000000000000000000000000000 P 98c29b9ddada432fb88d5bb8e561b2e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:32:00.714993 8049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98c29b9ddada432fb88d5bb8e561b2e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:32:00.715138 8049 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98c29b9ddada432fb88d5bb8e561b2e2 [term 1 LEADER]: Becoming Leader. State: Replica: 98c29b9ddada432fb88d5bb8e561b2e2, State: Running, Role: LEADER
I20260619 20:32:00.715195 8044 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98c29b9ddada432fb88d5bb8e561b2e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:32:00.715298 8049 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98c29b9ddada432fb88d5bb8e561b2e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98c29b9ddada432fb88d5bb8e561b2e2" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 33041 } }
I20260619 20:32:00.715709 8050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98c29b9ddada432fb88d5bb8e561b2e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98c29b9ddada432fb88d5bb8e561b2e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98c29b9ddada432fb88d5bb8e561b2e2" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 33041 } } }
I20260619 20:32:00.715798 8050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98c29b9ddada432fb88d5bb8e561b2e2 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:00.716032 8051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98c29b9ddada432fb88d5bb8e561b2e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98c29b9ddada432fb88d5bb8e561b2e2. Latest consensus state: current_term: 1 leader_uuid: "98c29b9ddada432fb88d5bb8e561b2e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98c29b9ddada432fb88d5bb8e561b2e2" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 33041 } } }
I20260619 20:32:00.716132 8051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98c29b9ddada432fb88d5bb8e561b2e2 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:00.716290 8057 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:32:00.716813 8057 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 20:32:00 dist-test-slave-4m58 krb5kdc[7963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:32:00.720316 8057 catalog_manager.cc:1347] Generated new cluster ID: 9d6b33492fd94240bbfb3d6ef5934d6c
I20260619 20:32:00.720366 8057 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:32:00.724838 8048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:00.713911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55936 (local address 127.4.83.126:33041)
0619 20:32:00.714488 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:00.714498 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:00.714522 (+ 24us) server_negotiation.cc:407] Connection header received
0619 20:32:00.714633 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:00.714639 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:00.714688 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:00.714814 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:00.715528 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:00.716847 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:00.717697 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:00.718303 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:00.718413 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:00.720860 (+ 2447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:00.720863 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:00.720864 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:00.720875 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:00.723209 (+ 2334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:00.723615 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:00.723617 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:00.723618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:00.723724 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:00.723939 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:00.723940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:00.723940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:00.724230 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:00.724373 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:00.724487 (+ 114us) server_negotiation.cc:299] Negotiation successful
0619 20:32:00.724620 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":489,"threads_started":1}
I20260619 20:32:00.733229 8057 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:32:00.733839 8057 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:32:00.737859 8057 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98c29b9ddada432fb88d5bb8e561b2e2: Generated new TSK 0
I20260619 20:32:00.738042 8057 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.83.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
I20260619 20:32:00.764458 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:0
--local_ip_for_outbound_sockets=127.4.83.65
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:33041
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:43873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:00.846246 8070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:00.846478 8070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:00.846503 8070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:00.847767 8070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:00.847823 8070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:00.847880 8070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:32:00.849587 8070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:43873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:33041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:00.849853 8070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:00.850106 8070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:00.855201 8075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:00.855288 8078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:00.855253 8076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:00.855515 8070 server_base.cc:1034] running on GCE node
I20260619 20:32:00.855749 8070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:00.856010 8070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:00.857339 8070 hybrid_clock.cc:648] HybridClock initialized: now 1781901120857296 us; error 61 us; skew 500 ppm
Jun 19 20:32:00 dist-test-slave-4m58 krb5kdc[7963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901120, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:00.860481 8070 init.cc:377] Logged in from keytab as kudu/127.4.83.65@KRBTEST.COM (short username kudu)
I20260619 20:32:00.860921 8070 webserver.cc:460] Webserver started at http://127.4.83.65:41417/ using document root <none> and password file <none>
I20260619 20:32:00.861122 8070 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:00.861168 8070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:00.861280 8070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:00.862317 8070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-0/data/instance:
uuid: "a4cbdc02da8d446b974ab44628b7ab8f"
format_stamp: "Formatted at 2026-06-19 20:32:00 on dist-test-slave-4m58"
server_key: "0feb20e9077b187fd11ab9059c8947ee"
server_key_iv: "553a83a7668afe5104e2c88e7f775715"
server_key_version: "encryptionkey@0"
I20260619 20:32:00.862637 8070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance:
uuid: "a4cbdc02da8d446b974ab44628b7ab8f"
format_stamp: "Formatted at 2026-06-19 20:32:00 on dist-test-slave-4m58"
server_key: "0feb20e9077b187fd11ab9059c8947ee"
server_key_iv: "553a83a7668afe5104e2c88e7f775715"
server_key_version: "encryptionkey@0"
I20260619 20:32:00.864034 8070 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:32:00.864826 8085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:00.865021 8070 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:00.865114 8070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "a4cbdc02da8d446b974ab44628b7ab8f"
format_stamp: "Formatted at 2026-06-19 20:32:00 on dist-test-slave-4m58"
server_key: "0feb20e9077b187fd11ab9059c8947ee"
server_key_iv: "553a83a7668afe5104e2c88e7f775715"
server_key_version: "encryptionkey@0"
I20260619 20:32:00.865190 8070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:00.891726 8070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:00.894402 8070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:00.894546 8070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:00.894862 8070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:00.895180 8070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:00.895212 8070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:00.895236 8070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:00.895248 8070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:00.900875 8070 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:36635
I20260619 20:32:00.900951 8198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:36635 every 8 connection(s)
I20260619 20:32:00.901232 8070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
Jun 19 20:32:00 dist-test-slave-4m58 krb5kdc[7963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901120, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:32:00.909699 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 8070
I20260619 20:32:00.909818 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance
I20260619 20:32:00.910025 4429 external_mini_cluster.cc:1428] Setting key 25c10ac32d513255fb30932fb6a36dc4
I20260619 20:32:00.911264 8048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:00.901970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:52329 (local address 127.4.83.126:33041)
0619 20:32:00.902034 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:00.902036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:00.902290 (+ 254us) server_negotiation.cc:407] Connection header received
0619 20:32:00.903176 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:00.903177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:00.903192 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:00.903261 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:00.904073 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:00.904606 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:00.905663 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:00.905887 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:00.905953 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:00.908380 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:00.908381 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:00.908382 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:00.908392 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:00.909720 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:00.910272 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:00.910274 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:00.910274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:00.910350 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:00.910589 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:00.910590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:00.910590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:00.910734 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:00.910794 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:00.911073 (+ 279us) server_negotiation.cc:299] Negotiation successful
0619 20:32:00.911176 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 20:32:00.911434 8201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:00.901929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:33041 (local address 127.4.83.65:52329)
0619 20:32:00.902112 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:00.902156 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:00.903078 (+ 922us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:00.903345 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:00.903349 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:00.903618 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:00.903820 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:00.903824 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:00.904709 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:00.904710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:00.905567 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:00.905569 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:00.906004 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:00.906005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:00.906125 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:00.906744 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:00.906753 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:00.908282 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:00.909842 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:00.909852 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:00.909854 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:00.910174 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:00.910413 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:00.910414 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:00.910414 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:00.910530 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:00.910793 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:00.910800 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:00.910943 (+ 143us) client_negotiation.cc:769] Sending connection context
0619 20:32:00.911036 (+ 93us) client_negotiation.cc:240] Negotiation successful
0619 20:32:00.911207 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":98,"threads_started":1}
I20260619 20:32:00.912257 8199 heartbeater.cc:347] Connected to a master server at 127.4.83.126:33041
I20260619 20:32:00.912353 8199 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:00.912539 8199 heartbeater.cc:511] Master 127.4.83.126:33041 requested a full tablet report, sending...
I20260619 20:32:00.913038 8008 ts_manager.cc:194] Registered new tserver with Master: a4cbdc02da8d446b974ab44628b7ab8f (127.4.83.65:36635)
I20260619 20:32:00.913653 8008 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.65@KRBTEST.COM'} at 127.4.83.65:52329
WARNING: no policy specified for kudu/127.4.83.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
I20260619 20:32:00.939219 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:0
--local_ip_for_outbound_sockets=127.4.83.66
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:33041
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:43873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:01.018662 8204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:01.018882 8204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:01.018914 8204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:01.020102 8204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:01.020156 8204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:01.020205 8204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:32:01.021791 8204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:43873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:33041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:01.022042 8204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:01.022258 8204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:01.027138 8209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:01.027251 8204 server_base.cc:1034] running on GCE node
W20260619 20:32:01.027206 8212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:01.027108 8210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:01.027559 8204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:01.027796 8204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:01.028952 8204 hybrid_clock.cc:648] HybridClock initialized: now 1781901121028941 us; error 32 us; skew 500 ppm
Jun 19 20:32:01 dist-test-slave-4m58 krb5kdc[7963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901121, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:01.031421 8204 init.cc:377] Logged in from keytab as kudu/127.4.83.66@KRBTEST.COM (short username kudu)
I20260619 20:32:01.031908 8204 webserver.cc:460] Webserver started at http://127.4.83.66:45519/ using document root <none> and password file <none>
I20260619 20:32:01.032106 8204 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:01.032146 8204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:01.032243 8204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:01.033255 8204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-1/data/instance:
uuid: "1d765f15df4e48e1bf84d19a8bc68bb1"
format_stamp: "Formatted at 2026-06-19 20:32:01 on dist-test-slave-4m58"
server_key: "b09f5a60b42a08ace05e84095e08d94f"
server_key_iv: "105ca8a1bd2758bbed56a173d68770f8"
server_key_version: "encryptionkey@0"
I20260619 20:32:01.033610 8204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance:
uuid: "1d765f15df4e48e1bf84d19a8bc68bb1"
format_stamp: "Formatted at 2026-06-19 20:32:01 on dist-test-slave-4m58"
server_key: "b09f5a60b42a08ace05e84095e08d94f"
server_key_iv: "105ca8a1bd2758bbed56a173d68770f8"
server_key_version: "encryptionkey@0"
I20260619 20:32:01.034878 8204 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:32:01.035552 8219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:01.035720 8204 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:32:01.035791 8204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "1d765f15df4e48e1bf84d19a8bc68bb1"
format_stamp: "Formatted at 2026-06-19 20:32:01 on dist-test-slave-4m58"
server_key: "b09f5a60b42a08ace05e84095e08d94f"
server_key_iv: "105ca8a1bd2758bbed56a173d68770f8"
server_key_version: "encryptionkey@0"
I20260619 20:32:01.035871 8204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:01.054149 8204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:01.056779 8204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:01.057128 8204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:01.057375 8204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:01.057665 8204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:01.057696 8204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:01.057718 8204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:01.057730 8204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:01.063652 8204 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:38483
I20260619 20:32:01.063720 8332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:38483 every 8 connection(s)
I20260619 20:32:01.064255 8204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
Jun 19 20:32:01 dist-test-slave-4m58 krb5kdc[7963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901121, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:32:01.074349 8048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:01.065041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:52919 (local address 127.4.83.126:33041)
0619 20:32:01.065124 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:01.065126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:01.065289 (+ 163us) server_negotiation.cc:407] Connection header received
0619 20:32:01.066267 (+ 978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:01.066269 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:01.066283 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:01.066335 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:01.067000 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:01.067526 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:01.068559 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:01.068753 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:01.068810 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:01.071142 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:01.071144 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:01.071144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:01.071152 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:01.072713 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:01.073234 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:01.073236 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:01.073236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:01.073300 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:01.073620 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:01.073622 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:01.073622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:01.073767 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:01.073839 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:01.074183 (+ 344us) server_negotiation.cc:299] Negotiation successful
0619 20:32:01.074274 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 20:32:01.074440 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 8204
I20260619 20:32:01.074543 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance
I20260619 20:32:01.074652 8335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:01.064965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:33041 (local address 127.4.83.66:52919)
0619 20:32:01.065164 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:01.065211 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:01.066158 (+ 947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:01.066406 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:01.066411 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:01.066705 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:01.066907 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:01.066911 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:01.067609 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:01.067610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:01.068475 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:01.068478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:01.068825 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:01.068826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:01.068951 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:01.069474 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:01.069482 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:01.071017 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:01.072812 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:01.072822 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:01.072824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:01.073138 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:01.073389 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:01.073390 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:01.073390 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:01.073524 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:01.073851 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:01.073858 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:01.074002 (+ 144us) client_negotiation.cc:769] Sending connection context
0619 20:32:01.074118 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 20:32:01.074310 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":85,"threads_started":1}
I20260619 20:32:01.074776 4429 external_mini_cluster.cc:1428] Setting key 9ab5704a9e002286ca74ae237422f365
I20260619 20:32:01.075170 8333 heartbeater.cc:347] Connected to a master server at 127.4.83.126:33041
I20260619 20:32:01.075286 8333 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:01.075488 8333 heartbeater.cc:511] Master 127.4.83.126:33041 requested a full tablet report, sending...
I20260619 20:32:01.075946 8008 ts_manager.cc:194] Registered new tserver with Master: 1d765f15df4e48e1bf84d19a8bc68bb1 (127.4.83.66:38483)
I20260619 20:32:01.076331 8008 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.66@KRBTEST.COM'} at 127.4.83.66:52919
WARNING: no policy specified for kudu/127.4.83.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
I20260619 20:32:01.100957 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:0
--local_ip_for_outbound_sockets=127.4.83.67
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:33041
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:43873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:01.183481 8338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:01.183704 8338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:01.183730 8338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:01.185010 8338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:01.185058 8338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:01.185087 8338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:32:01.186677 8338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:43873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:33041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:01.186911 8338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:01.187134 8338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:01.192055 8346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:01.192055 8343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:01.192055 8344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:01.192554 8338 server_base.cc:1034] running on GCE node
I20260619 20:32:01.192796 8338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:01.193043 8338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:01.194185 8338 hybrid_clock.cc:648] HybridClock initialized: now 1781901121194171 us; error 33 us; skew 500 ppm
Jun 19 20:32:01 dist-test-slave-4m58 krb5kdc[7963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901121, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:01.196679 8338 init.cc:377] Logged in from keytab as kudu/127.4.83.67@KRBTEST.COM (short username kudu)
I20260619 20:32:01.197156 8338 webserver.cc:460] Webserver started at http://127.4.83.67:42475/ using document root <none> and password file <none>
I20260619 20:32:01.197360 8338 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:01.197407 8338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:01.197532 8338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:01.198566 8338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-2/data/instance:
uuid: "ff7767d87a3a430195d0029ba4f05581"
format_stamp: "Formatted at 2026-06-19 20:32:01 on dist-test-slave-4m58"
server_key: "2a793866a63f75294ecebfd29aab9810"
server_key_iv: "85bf334c72bd64903ef7f861ceef6513"
server_key_version: "encryptionkey@0"
I20260619 20:32:01.198881 8338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance:
uuid: "ff7767d87a3a430195d0029ba4f05581"
format_stamp: "Formatted at 2026-06-19 20:32:01 on dist-test-slave-4m58"
server_key: "2a793866a63f75294ecebfd29aab9810"
server_key_iv: "85bf334c72bd64903ef7f861ceef6513"
server_key_version: "encryptionkey@0"
I20260619 20:32:01.200141 8338 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:32:01.200786 8353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:01.200937 8338 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:32:01.201013 8338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "ff7767d87a3a430195d0029ba4f05581"
format_stamp: "Formatted at 2026-06-19 20:32:01 on dist-test-slave-4m58"
server_key: "2a793866a63f75294ecebfd29aab9810"
server_key_iv: "85bf334c72bd64903ef7f861ceef6513"
server_key_version: "encryptionkey@0"
I20260619 20:32:01.201073 8338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:01.213654 8338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:01.215762 8338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:01.215934 8338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:01.216136 8338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:01.216447 8338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:01.216485 8338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:01.216516 8338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:01.216542 8338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:01.222045 8338 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:43149
I20260619 20:32:01.222090 8466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:43149 every 8 connection(s)
I20260619 20:32:01.222400 8338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
I20260619 20:32:01.226007 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 8338
I20260619 20:32:01.226125 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance
I20260619 20:32:01.226313 4429 external_mini_cluster.cc:1428] Setting key 0053124c8c155f0364e495f8b081b23a
Jun 19 20:32:01 dist-test-slave-4m58 krb5kdc[7963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901121, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:32:01.231763 8048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:01.223079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:46593 (local address 127.4.83.126:33041)
0619 20:32:01.223149 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:01.223151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:01.223333 (+ 182us) server_negotiation.cc:407] Connection header received
0619 20:32:01.224311 (+ 978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:01.224313 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:01.224325 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:01.224386 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:01.225007 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:01.225514 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:01.226756 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:01.227012 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:01.227072 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:01.229146 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:01.229148 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:01.229148 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:01.229156 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:01.230439 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:01.230843 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:01.230845 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:01.230845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:01.230905 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:01.231123 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:01.231124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:01.231124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:01.231251 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:01.231308 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:01.231582 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 20:32:01.231669 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 20:32:01.231977 8469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:01.223038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:33041 (local address 127.4.83.67:46593)
0619 20:32:01.223196 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:01.223234 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:01.224208 (+ 974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:01.224454 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:01.224457 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:01.224721 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:01.224917 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:01.224921 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:01.225602 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:01.225603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:01.226672 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:01.226675 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:01.227090 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:01.227091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:01.227207 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:01.227650 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:01.227658 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:01.229046 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:01.230523 (+ 1477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:01.230532 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:01.230535 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:01.230764 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:01.230960 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:01.230961 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:01.230961 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:01.231066 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:01.231308 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:01.231314 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:01.231459 (+ 145us) client_negotiation.cc:769] Sending connection context
0619 20:32:01.231551 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 20:32:01.231720 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":75,"threads_started":1}
I20260619 20:32:01.232477 8467 heartbeater.cc:347] Connected to a master server at 127.4.83.126:33041
I20260619 20:32:01.232573 8467 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:01.232750 8467 heartbeater.cc:511] Master 127.4.83.126:33041 requested a full tablet report, sending...
I20260619 20:32:01.233089 8008 ts_manager.cc:194] Registered new tserver with Master: ff7767d87a3a430195d0029ba4f05581 (127.4.83.67:43149)
I20260619 20:32:01.233457 8008 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.67@KRBTEST.COM'} at 127.4.83.67:46593
I20260619 20:32:01.238137 4429 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 20:32:01 dist-test-slave-4m58 krb5kdc[7963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 20:32:01 dist-test-slave-4m58 krb5kdc[7963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.65@KRBTEST.COM
I20260619 20:32:01.260969 8474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:01.252982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43922 (local address 127.4.83.65:36635)
0619 20:32:01.253190 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:01.253194 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:01.253207 (+ 13us) server_negotiation.cc:407] Connection header received
0619 20:32:01.253235 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:01.253237 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:01.253251 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:01.253322 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:01.253868 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:01.254618 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:01.255365 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:01.255698 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:01.255760 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:01.258050 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:01.258059 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:01.258060 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:01.258068 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:01.259751 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:01.260181 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:01.260186 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:01.260186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:01.260264 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:01.260411 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:01.260411 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:01.260412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:01.260556 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:01.260697 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:01.260794 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 20:32:01.260869 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":99,"threads_started":1}
Jun 19 20:32:01 dist-test-slave-4m58 krb5kdc[7963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:32:01.268833 8048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:01.262276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55952 (local address 127.4.83.126:33041)
0619 20:32:01.262342 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:01.262344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:01.262353 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:32:01.262380 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:01.262381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:01.262391 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:01.262437 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:01.262861 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:01.263355 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:01.264046 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:01.264235 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:01.264288 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:01.266492 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:01.266494 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:01.266494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:01.266502 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:01.267694 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:01.268051 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:01.268052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:01.268053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:01.268105 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:01.268265 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:01.268265 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:01.268266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:01.268459 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:01.268548 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:01.268649 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 20:32:01.268729 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260619 20:32:01.269090 8008 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55952
I20260619 20:32:01.269416 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 8070
I20260619 20:32:01.273411 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 8204
I20260619 20:32:01.279178 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 8338
I20260619 20:32:01.282793 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 7976
2026-06-19T20:32:01Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1770 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:32:01 dist-test-slave-4m58 krb5kdc[8479](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:32:01 dist-test-slave-4m58 krb5kdc[8479](info): set up 2 sockets
Jun 19 20:32:01 dist-test-slave-4m58 krb5kdc[8479](info): commencing operation
krb5kdc: starting...
W20260619 20:32:02.249037 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.948s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:32:02 dist-test-slave-4m58 krb5kdc[8479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:32:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:32:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
I20260619 20:32:02.371766 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:37801
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:37801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:02.457029 8492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:02.457281 8492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:02.457320 8492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:02.458667 8492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:32:02.458726 8492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:02.458746 8492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:02.458765 8492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:32:02.458791 8492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:32:02.460606 8492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:37801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:37801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:02.460939 8492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:02.461313 8492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:02.467002 8500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:02.467017 8498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:02.467017 8497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:02.467053 8492 server_base.cc:1034] running on GCE node
I20260619 20:32:02.467515 8492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:02.467921 8492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:02.469295 8492 hybrid_clock.cc:648] HybridClock initialized: now 1781901122469229 us; error 54 us; skew 500 ppm
Jun 19 20:32:02 dist-test-slave-4m58 krb5kdc[8479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901122, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:02.472882 8492 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:32:02.474619 8492 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:02.474733 8492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:02.474850 8492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:02.476328 8492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "ac256e9a1e8d47ac984cb483569db7d8"
format_stamp: "Formatted at 2026-06-19 20:32:02 on dist-test-slave-4m58"
server_key: "227f3ffd425d3011c46076c1f0edb71e"
server_key_iv: "87d8b207976ccf4df9200f9171b3d3cb"
server_key_version: "encryptionkey@0"
I20260619 20:32:02.476716 8492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "ac256e9a1e8d47ac984cb483569db7d8"
format_stamp: "Formatted at 2026-06-19 20:32:02 on dist-test-slave-4m58"
server_key: "227f3ffd425d3011c46076c1f0edb71e"
server_key_iv: "87d8b207976ccf4df9200f9171b3d3cb"
server_key_version: "encryptionkey@0"
I20260619 20:32:02.478161 8492 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:32:02.479081 8506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:02.479519 8492 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:02.479682 8492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "ac256e9a1e8d47ac984cb483569db7d8"
format_stamp: "Formatted at 2026-06-19 20:32:02 on dist-test-slave-4m58"
server_key: "227f3ffd425d3011c46076c1f0edb71e"
server_key_iv: "87d8b207976ccf4df9200f9171b3d3cb"
server_key_version: "encryptionkey@0"
I20260619 20:32:02.479794 8492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:02.498474 8492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:02.501031 8492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:02.501161 8492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:02.505843 8492 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:37801
I20260619 20:32:02.505844 8558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:37801 every 8 connection(s)
I20260619 20:32:02.506253 8492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:32:02.507285 8559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:02.507433 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 8492
I20260619 20:32:02.507534 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:32:02.507748 4429 external_mini_cluster.cc:1428] Setting key 085515d768771a3bee4a5cebdac79d34
I20260619 20:32:02.510812 8559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac256e9a1e8d47ac984cb483569db7d8: Bootstrap starting.
I20260619 20:32:02.511552 8559 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac256e9a1e8d47ac984cb483569db7d8: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:02.511953 8559 log.cc:824] T 00000000000000000000000000000000 P ac256e9a1e8d47ac984cb483569db7d8: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:02.512658 8559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac256e9a1e8d47ac984cb483569db7d8: No bootstrap required, opened a new log
Jun 19 20:32:02 dist-test-slave-4m58 krb5kdc[8479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:32:02.515241 8559 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac256e9a1e8d47ac984cb483569db7d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac256e9a1e8d47ac984cb483569db7d8" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37801 } }
I20260619 20:32:02.515440 8559 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac256e9a1e8d47ac984cb483569db7d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:02.515496 8559 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac256e9a1e8d47ac984cb483569db7d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac256e9a1e8d47ac984cb483569db7d8, State: Initialized, Role: FOLLOWER
I20260619 20:32:02.515661 8559 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac256e9a1e8d47ac984cb483569db7d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac256e9a1e8d47ac984cb483569db7d8" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37801 } }
I20260619 20:32:02.515753 8559 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac256e9a1e8d47ac984cb483569db7d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:32:02.515794 8559 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac256e9a1e8d47ac984cb483569db7d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:32:02.515870 8559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac256e9a1e8d47ac984cb483569db7d8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:02.516502 8559 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac256e9a1e8d47ac984cb483569db7d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac256e9a1e8d47ac984cb483569db7d8" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37801 } }
I20260619 20:32:02.516609 8559 leader_election.cc:302] T 00000000000000000000000000000000 P ac256e9a1e8d47ac984cb483569db7d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac256e9a1e8d47ac984cb483569db7d8; no voters:
I20260619 20:32:02.516781 8559 leader_election.cc:288] T 00000000000000000000000000000000 P ac256e9a1e8d47ac984cb483569db7d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:32:02.516896 8564 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac256e9a1e8d47ac984cb483569db7d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:32:02.517045 8559 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac256e9a1e8d47ac984cb483569db7d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:32:02.517107 8564 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac256e9a1e8d47ac984cb483569db7d8 [term 1 LEADER]: Becoming Leader. State: Replica: ac256e9a1e8d47ac984cb483569db7d8, State: Running, Role: LEADER
I20260619 20:32:02.517280 8564 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac256e9a1e8d47ac984cb483569db7d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac256e9a1e8d47ac984cb483569db7d8" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37801 } }
I20260619 20:32:02.517663 8565 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac256e9a1e8d47ac984cb483569db7d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac256e9a1e8d47ac984cb483569db7d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac256e9a1e8d47ac984cb483569db7d8" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37801 } } }
I20260619 20:32:02.517755 8565 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac256e9a1e8d47ac984cb483569db7d8 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:02.518124 8566 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac256e9a1e8d47ac984cb483569db7d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac256e9a1e8d47ac984cb483569db7d8. Latest consensus state: current_term: 1 leader_uuid: "ac256e9a1e8d47ac984cb483569db7d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac256e9a1e8d47ac984cb483569db7d8" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37801 } } }
I20260619 20:32:02.518183 8573 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:32:02.518204 8566 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac256e9a1e8d47ac984cb483569db7d8 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:02.518792 8573 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:32:02.520859 8573 catalog_manager.cc:1347] Generated new cluster ID: 1dc03a90b9084587898cd75104220071
I20260619 20:32:02.520920 8573 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:32:02.523618 8562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:02.508383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43894 (local address 127.4.83.126:37801)
0619 20:32:02.508566 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:02.508576 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:02.508610 (+ 34us) server_negotiation.cc:407] Connection header received
0619 20:32:02.509090 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:02.509101 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:02.509166 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:02.509334 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:02.509996 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:02.511350 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:02.512360 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:02.512825 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:02.512910 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:02.516079 (+ 3169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:02.516084 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:02.516085 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:02.516100 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:02.521283 (+ 5183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:02.521882 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:02.521885 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:02.521886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:02.522212 (+ 326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:02.522606 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:02.522607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:02.522607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:02.522954 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:02.523117 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:02.523291 (+ 174us) server_negotiation.cc:299] Negotiation successful
0619 20:32:02.523438 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
I20260619 20:32:02.536119 8573 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:32:02.536729 8573 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:32:02.546360 8573 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac256e9a1e8d47ac984cb483569db7d8: Generated new TSK 0
I20260619 20:32:02.546617 8573 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.83.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
I20260619 20:32:02.585714 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:0
--local_ip_for_outbound_sockets=127.4.83.65
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:37801
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:02.676620 8585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:02.676862 8585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:02.676889 8585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:02.678241 8585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:02.678299 8585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:02.678329 8585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:32:02.680109 8585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:37801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:02.680373 8585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:02.680610 8585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:02.686784 8591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:02.686920 8585 server_base.cc:1034] running on GCE node
W20260619 20:32:02.686789 8593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:02.686789 8590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:02.687314 8585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:02.687599 8585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:02.689028 8585 hybrid_clock.cc:648] HybridClock initialized: now 1781901122688996 us; error 55 us; skew 500 ppm
Jun 19 20:32:02 dist-test-slave-4m58 krb5kdc[8479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901122, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:02.697909 8585 init.cc:377] Logged in from keytab as kudu/127.4.83.65@KRBTEST.COM (short username kudu)
I20260619 20:32:02.698349 8585 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:02.698408 8585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:02.698545 8585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:02.699663 8585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-0/data/instance:
uuid: "aec52eb99aef4416b5d7c9a64ec945d3"
format_stamp: "Formatted at 2026-06-19 20:32:02 on dist-test-slave-4m58"
server_key: "58b3e7896627cbd5a545abff1e58d6da"
server_key_iv: "cd194705a2d06aeb9d9b1c4a777ecf3a"
server_key_version: "encryptionkey@0"
I20260619 20:32:02.700040 8585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance:
uuid: "aec52eb99aef4416b5d7c9a64ec945d3"
format_stamp: "Formatted at 2026-06-19 20:32:02 on dist-test-slave-4m58"
server_key: "58b3e7896627cbd5a545abff1e58d6da"
server_key_iv: "cd194705a2d06aeb9d9b1c4a777ecf3a"
server_key_version: "encryptionkey@0"
I20260619 20:32:02.701429 8585 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:02.702286 8599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:02.702509 8585 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:02.702607 8585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "aec52eb99aef4416b5d7c9a64ec945d3"
format_stamp: "Formatted at 2026-06-19 20:32:02 on dist-test-slave-4m58"
server_key: "58b3e7896627cbd5a545abff1e58d6da"
server_key_iv: "cd194705a2d06aeb9d9b1c4a777ecf3a"
server_key_version: "encryptionkey@0"
I20260619 20:32:02.702687 8585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:02.710748 8585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:02.713699 8585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:02.713860 8585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:02.714105 8585 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:02.714442 8585 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:02.714478 8585 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:02.714507 8585 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:02.714524 8585 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:02.721131 8585 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:45185
I20260619 20:32:02.721179 8712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:45185 every 8 connection(s)
I20260619 20:32:02.721458 8585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
Jun 19 20:32:02 dist-test-slave-4m58 krb5kdc[8479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901122, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:32:02.731372 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 8585
I20260619 20:32:02.731493 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance
I20260619 20:32:02.731685 4429 external_mini_cluster.cc:1428] Setting key 7299cda34c0de1ff8f6f81d53472fcf0
I20260619 20:32:02.732265 8562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:02.722194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:51743 (local address 127.4.83.126:37801)
0619 20:32:02.722260 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:02.722262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:02.722541 (+ 279us) server_negotiation.cc:407] Connection header received
0619 20:32:02.723142 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:02.723144 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:02.723155 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:02.723212 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:02.724244 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:02.724848 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:02.726229 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:02.726440 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:02.726508 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:02.729053 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:02.729055 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:02.729055 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:02.729064 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:02.730467 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:02.730981 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:02.730983 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:02.730983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:02.731047 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:02.731472 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:02.731473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:02.731473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:02.731619 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:02.731680 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:02.732065 (+ 385us) server_negotiation.cc:299] Negotiation successful
0619 20:32:02.732162 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 20:32:02.732441 8715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:02.722165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:37801 (local address 127.4.83.65:51743)
0619 20:32:02.722395 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:02.722435 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:02.723035 (+ 600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:02.723282 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:02.723285 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:02.723565 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:02.724169 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:02.724175 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:02.725020 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:02.725021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:02.725915 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:02.725918 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:02.726543 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:02.726544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:02.726672 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:02.727249 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:02.727258 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:02.728861 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:02.730578 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:02.730587 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:02.730589 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:02.730880 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:02.731176 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:02.731177 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:02.731178 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:02.731363 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:02.731684 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:02.731692 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:02.731927 (+ 235us) client_negotiation.cc:769] Sending connection context
0619 20:32:02.732030 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 20:32:02.732204 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":87,"threads_started":1}
I20260619 20:32:02.732908 8713 heartbeater.cc:347] Connected to a master server at 127.4.83.126:37801
I20260619 20:32:02.733008 8713 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:02.733163 8713 heartbeater.cc:511] Master 127.4.83.126:37801 requested a full tablet report, sending...
I20260619 20:32:02.733711 8523 ts_manager.cc:194] Registered new tserver with Master: aec52eb99aef4416b5d7c9a64ec945d3 (127.4.83.65:45185)
I20260619 20:32:02.734659 8523 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.65@KRBTEST.COM'} at 127.4.83.65:51743
WARNING: no policy specified for kudu/127.4.83.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
I20260619 20:32:02.760360 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:0
--local_ip_for_outbound_sockets=127.4.83.66
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:37801
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:02.839933 8718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:02.840166 8718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:02.840191 8718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:02.841339 8718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:02.841379 8718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:02.841406 8718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:32:02.842880 8718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:37801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:02.843093 8718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:02.843297 8718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:02.848484 8726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:02.848562 8724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:02.848685 8718 server_base.cc:1034] running on GCE node
W20260619 20:32:02.848584 8723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:02.848958 8718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:02.849221 8718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:02.850382 8718 hybrid_clock.cc:648] HybridClock initialized: now 1781901122850370 us; error 33 us; skew 500 ppm
Jun 19 20:32:02 dist-test-slave-4m58 krb5kdc[8479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901122, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:02.853008 8718 init.cc:377] Logged in from keytab as kudu/127.4.83.66@KRBTEST.COM (short username kudu)
I20260619 20:32:02.853319 8718 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:02.853368 8718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:02.853495 8718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:02.854522 8718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-1/data/instance:
uuid: "faacb878616b418b90a54723a418ecee"
format_stamp: "Formatted at 2026-06-19 20:32:02 on dist-test-slave-4m58"
server_key: "753eeb28adcc9569bfe652cfd11dee21"
server_key_iv: "3ad9f3b7bef59404b0abb0e0e4ac85da"
server_key_version: "encryptionkey@0"
I20260619 20:32:02.854871 8718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance:
uuid: "faacb878616b418b90a54723a418ecee"
format_stamp: "Formatted at 2026-06-19 20:32:02 on dist-test-slave-4m58"
server_key: "753eeb28adcc9569bfe652cfd11dee21"
server_key_iv: "3ad9f3b7bef59404b0abb0e0e4ac85da"
server_key_version: "encryptionkey@0"
I20260619 20:32:02.856184 8718 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:32:02.857136 8732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:02.857379 8718 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:02.857472 8718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "faacb878616b418b90a54723a418ecee"
format_stamp: "Formatted at 2026-06-19 20:32:02 on dist-test-slave-4m58"
server_key: "753eeb28adcc9569bfe652cfd11dee21"
server_key_iv: "3ad9f3b7bef59404b0abb0e0e4ac85da"
server_key_version: "encryptionkey@0"
I20260619 20:32:02.857540 8718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:02.873340 8718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:02.875510 8718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:02.875656 8718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:02.875954 8718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:02.876263 8718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:02.876296 8718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:02.876327 8718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:02.876343 8718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:02.885511 8718 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:43437
I20260619 20:32:02.885561 8845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:43437 every 8 connection(s)
I20260619 20:32:02.885982 8718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
Jun 19 20:32:02 dist-test-slave-4m58 krb5kdc[8479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901122, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:32:02.895488 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 8718
I20260619 20:32:02.895579 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance
I20260619 20:32:02.895784 4429 external_mini_cluster.cc:1428] Setting key 5f14c10287e6bf4395cc78e5fb37c40b
I20260619 20:32:02.895903 8562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:02.886824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:38967 (local address 127.4.83.126:37801)
0619 20:32:02.886989 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:02.886991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:02.887176 (+ 185us) server_negotiation.cc:407] Connection header received
0619 20:32:02.887746 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:02.887747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:02.887760 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:02.887807 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:02.888801 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:02.889306 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:02.890307 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:02.890514 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:02.890571 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:02.892819 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:02.892821 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:02.892821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:02.892829 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:02.894358 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:02.894826 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:02.894827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:02.894827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:02.894953 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:02.895220 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:02.895221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:02.895221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:02.895355 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:02.895412 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:02.895692 (+ 280us) server_negotiation.cc:299] Negotiation successful
0619 20:32:02.895780 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260619 20:32:02.896109 8848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:02.886798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:37801 (local address 127.4.83.66:38967)
0619 20:32:02.887045 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:02.887086 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:02.887614 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:02.887920 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:02.887924 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:02.888216 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:02.888702 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:02.888707 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:02.889392 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:02.889393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:02.890221 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:02.890224 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:02.890588 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:02.890589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:02.890713 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:02.891245 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:02.891254 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:02.892717 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:02.894460 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:02.894470 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:02.894472 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:02.894752 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:02.895037 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:02.895038 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:02.895039 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:02.895152 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:02.895410 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:02.895416 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:02.895557 (+ 141us) client_negotiation.cc:769] Sending connection context
0619 20:32:02.895659 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 20:32:02.895866 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":105,"threads_started":1}
I20260619 20:32:02.896661 8846 heartbeater.cc:347] Connected to a master server at 127.4.83.126:37801
I20260619 20:32:02.896762 8846 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:02.896914 8846 heartbeater.cc:511] Master 127.4.83.126:37801 requested a full tablet report, sending...
I20260619 20:32:02.897258 8523 ts_manager.cc:194] Registered new tserver with Master: faacb878616b418b90a54723a418ecee (127.4.83.66:43437)
I20260619 20:32:02.897625 8523 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.66@KRBTEST.COM'} at 127.4.83.66:38967
WARNING: no policy specified for kudu/127.4.83.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
I20260619 20:32:02.929735 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:0
--local_ip_for_outbound_sockets=127.4.83.67
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:37801
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:03.023566 8851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:03.023820 8851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:03.023907 8851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:03.026110 8851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:03.026190 8851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:03.026235 8851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:32:03.028221 8851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:37801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:03.028469 8851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:03.028671 8851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:03.033638 8856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:03.033630 8859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:03.033630 8857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:03.033874 8851 server_base.cc:1034] running on GCE node
I20260619 20:32:03.034085 8851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:03.034333 8851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:03.035486 8851 hybrid_clock.cc:648] HybridClock initialized: now 1781901123035467 us; error 34 us; skew 500 ppm
Jun 19 20:32:03 dist-test-slave-4m58 krb5kdc[8479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901123, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:03.038141 8851 init.cc:377] Logged in from keytab as kudu/127.4.83.67@KRBTEST.COM (short username kudu)
I20260619 20:32:03.038496 8851 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:03.038543 8851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:03.038642 8851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:03.039666 8851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-2/data/instance:
uuid: "672f7b7ab39543f99858fcd65804246b"
format_stamp: "Formatted at 2026-06-19 20:32:03 on dist-test-slave-4m58"
server_key: "c69ef118278fb5cfc13bb9a7f65fe8ba"
server_key_iv: "f7d0250606cba8cb5b489d3b9fcc7c74"
server_key_version: "encryptionkey@0"
I20260619 20:32:03.040005 8851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance:
uuid: "672f7b7ab39543f99858fcd65804246b"
format_stamp: "Formatted at 2026-06-19 20:32:03 on dist-test-slave-4m58"
server_key: "c69ef118278fb5cfc13bb9a7f65fe8ba"
server_key_iv: "f7d0250606cba8cb5b489d3b9fcc7c74"
server_key_version: "encryptionkey@0"
I20260619 20:32:03.041201 8851 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:32:03.041972 8865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:03.042140 8851 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:32:03.042223 8851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "672f7b7ab39543f99858fcd65804246b"
format_stamp: "Formatted at 2026-06-19 20:32:03 on dist-test-slave-4m58"
server_key: "c69ef118278fb5cfc13bb9a7f65fe8ba"
server_key_iv: "f7d0250606cba8cb5b489d3b9fcc7c74"
server_key_version: "encryptionkey@0"
I20260619 20:32:03.042301 8851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:03.075801 8851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:03.078285 8851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:03.078416 8851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:03.078693 8851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:03.079015 8851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:03.079056 8851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:03.079082 8851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:03.079095 8851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:03.085846 8851 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:37401
I20260619 20:32:03.085927 8978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:37401 every 8 connection(s)
I20260619 20:32:03.086187 8851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
Jun 19 20:32:03 dist-test-slave-4m58 krb5kdc[8479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901123, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:32:03.095928 8562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:03.086896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:45713 (local address 127.4.83.126:37801)
0619 20:32:03.086941 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:03.086943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:03.087271 (+ 328us) server_negotiation.cc:407] Connection header received
0619 20:32:03.087797 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:03.087799 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:03.087811 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:03.087889 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:03.088883 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:03.089401 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:03.090402 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:03.090607 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:03.090663 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:03.093137 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:03.093140 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:03.093140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:03.093148 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:03.094558 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:03.094994 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:03.094996 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:03.094996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:03.095049 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:03.095265 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:03.095266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:03.095266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:03.095389 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:03.095445 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:03.095713 (+ 268us) server_negotiation.cc:299] Negotiation successful
0619 20:32:03.095798 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:32:03.096122 8981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:03.086923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:37801 (local address 127.4.83.67:45713)
0619 20:32:03.087113 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:03.087165 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:03.087695 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:03.087966 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:03.087971 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:03.088267 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:03.088786 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:03.088791 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:03.089491 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:03.089492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:03.090316 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:03.090318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:03.090679 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:03.090680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:03.090803 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:03.091403 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:03.091412 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:03.093026 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:03.094660 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:03.094669 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:03.094672 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:03.094921 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:03.095102 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:03.095103 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:03.095103 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:03.095209 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:03.095446 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:03.095453 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:03.095591 (+ 138us) client_negotiation.cc:769] Sending connection context
0619 20:32:03.095686 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 20:32:03.095899 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":70,"threads_started":1}
I20260619 20:32:03.096190 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 8851
I20260619 20:32:03.096295 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance
I20260619 20:32:03.096488 4429 external_mini_cluster.cc:1428] Setting key ecb4db320da59fe5eb11938ddc75c290
I20260619 20:32:03.096598 8979 heartbeater.cc:347] Connected to a master server at 127.4.83.126:37801
I20260619 20:32:03.096709 8979 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:03.096910 8979 heartbeater.cc:511] Master 127.4.83.126:37801 requested a full tablet report, sending...
I20260619 20:32:03.097311 8523 ts_manager.cc:194] Registered new tserver with Master: 672f7b7ab39543f99858fcd65804246b (127.4.83.67:37401)
I20260619 20:32:03.097699 8523 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.67@KRBTEST.COM'} at 127.4.83.67:45713
I20260619 20:32:03.108399 4429 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:32:03.115935 8562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:03.109744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43906 (local address 127.4.83.126:37801)
0619 20:32:03.109833 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:03.109835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:03.109902 (+ 67us) server_negotiation.cc:407] Connection header received
0619 20:32:03.109951 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:03.109952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:03.109961 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:03.110002 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:03.110609 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:03.111114 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:03.111914 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:03.112101 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:03.112167 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:03.113477 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:03.113478 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:03.113478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:03.113485 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:03.114914 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:03.115237 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:03.115237 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:03.115238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:03.115290 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:03.115478 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:03.115479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:03.115479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:03.115623 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:03.115698 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:03.115774 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 20:32:03.115861 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 20:32:03.117954 8523 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43906:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:32:03.119590 8523 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:32:03.127149 8990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:03.122192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:37401 (local address 127.0.0.1:54674)
0619 20:32:03.122355 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:03.122394 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:03.122459 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:03.123345 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:03.123346 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:03.123354 (+ 8us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:03.123416 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:03.123418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:03.124333 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:03.124334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:03.125999 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:03.126003 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:03.126806 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:03.126807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:03.126906 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:03.126922 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 20:32:03.126988 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 20:32:03.127058 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":59,"thread_start_us":56,"threads_started":1}
I20260619 20:32:03.127157 8993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:03.122366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54674 (local address 127.4.83.67:37401)
0619 20:32:03.122832 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:03.122836 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:03.122843 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:32:03.122894 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:03.122895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:03.122933 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:03.123005 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:03.123494 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:03.124244 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:03.126093 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:03.126718 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:03.126790 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:03.126853 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:03.127018 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 20:32:03.127058 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":77,"threads_started":1}
I20260619 20:32:03.127348 8995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:03.122592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44210 (local address 127.4.83.66:43437)
0619 20:32:03.122732 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:03.122743 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:03.122756 (+ 13us) server_negotiation.cc:407] Connection header received
0619 20:32:03.122793 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:03.122795 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:03.122808 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:03.122877 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:03.123081 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:03.124273 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:03.126122 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:03.126831 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:03.126903 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:03.126958 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:03.127212 (+ 254us) server_negotiation.cc:299] Negotiation successful
0619 20:32:03.127263 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
I20260619 20:32:03.127346 8994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:03.122192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:45185 (local address 127.0.0.1:52948)
0619 20:32:03.122586 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:03.122594 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:03.122629 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:03.123277 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:03.123278 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:03.123280 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:03.123324 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:03.123326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:03.124561 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:03.124562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:03.126216 (+ 1654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:03.126219 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:03.127144 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:03.127145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:03.127212 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:03.127215 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:32:03.127244 (+ 29us) client_negotiation.cc:240] Negotiation successful
0619 20:32:03.127282 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":3,"thread_start_us":84,"threads_started":1}
I20260619 20:32:03.127648 8992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:03.122316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52948 (local address 127.4.83.65:45185)
0619 20:32:03.123136 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:03.123139 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:03.123147 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:32:03.123176 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:03.123177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:03.123190 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:03.123261 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:03.123415 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:03.124476 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:03.126315 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:03.127052 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:03.127425 (+ 373us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:03.127499 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:03.127525 (+ 26us) server_negotiation.cc:299] Negotiation successful
0619 20:32:03.127569 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":787,"thread_start_us":97,"threads_started":1}
I20260619 20:32:03.127522 8991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:03.122382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:43437 (local address 127.0.0.1:44210)
0619 20:32:03.122718 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:03.122724 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:03.122748 (+ 24us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:03.122913 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:03.122916 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:03.122919 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:03.123005 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:03.123008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:03.124393 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:03.124394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:03.126031 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:03.126033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:03.126996 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:03.126997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:03.127062 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:03.127068 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 20:32:03.127112 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 20:32:03.127164 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":10,"thread_start_us":70,"threads_started":1}
I20260619 20:32:03.128351 8780 tablet_service.cc:1467] Processing CreateTablet for tablet 33f1929e336e4938a7ac0d19dea72055 (DEFAULT_TABLE table=test-table [id=64aab6e10cdd412cae5bdf64f3ed8baa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:32:03.128377 8644 tablet_service.cc:1467] Processing CreateTablet for tablet 33f1929e336e4938a7ac0d19dea72055 (DEFAULT_TABLE table=test-table [id=64aab6e10cdd412cae5bdf64f3ed8baa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:32:03.128381 8912 tablet_service.cc:1467] Processing CreateTablet for tablet 33f1929e336e4938a7ac0d19dea72055 (DEFAULT_TABLE table=test-table [id=64aab6e10cdd412cae5bdf64f3ed8baa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:32:03.128700 8780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33f1929e336e4938a7ac0d19dea72055. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:03.128700 8912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33f1929e336e4938a7ac0d19dea72055. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:03.128700 8644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33f1929e336e4938a7ac0d19dea72055. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:03.131300 8996 tablet_bootstrap.cc:492] T 33f1929e336e4938a7ac0d19dea72055 P faacb878616b418b90a54723a418ecee: Bootstrap starting.
I20260619 20:32:03.131867 8998 tablet_bootstrap.cc:492] T 33f1929e336e4938a7ac0d19dea72055 P 672f7b7ab39543f99858fcd65804246b: Bootstrap starting.
I20260619 20:32:03.132467 8996 tablet_bootstrap.cc:654] T 33f1929e336e4938a7ac0d19dea72055 P faacb878616b418b90a54723a418ecee: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:03.132745 8998 tablet_bootstrap.cc:654] T 33f1929e336e4938a7ac0d19dea72055 P 672f7b7ab39543f99858fcd65804246b: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:03.133198 8996 log.cc:824] T 33f1929e336e4938a7ac0d19dea72055 P faacb878616b418b90a54723a418ecee: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:03.133333 8998 log.cc:824] T 33f1929e336e4938a7ac0d19dea72055 P 672f7b7ab39543f99858fcd65804246b: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:03.133718 8997 tablet_bootstrap.cc:492] T 33f1929e336e4938a7ac0d19dea72055 P aec52eb99aef4416b5d7c9a64ec945d3: Bootstrap starting.
I20260619 20:32:03.134066 8996 tablet_bootstrap.cc:492] T 33f1929e336e4938a7ac0d19dea72055 P faacb878616b418b90a54723a418ecee: No bootstrap required, opened a new log
I20260619 20:32:03.134143 8996 ts_tablet_manager.cc:1397] T 33f1929e336e4938a7ac0d19dea72055 P faacb878616b418b90a54723a418ecee: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 20:32:03.134380 8997 tablet_bootstrap.cc:654] T 33f1929e336e4938a7ac0d19dea72055 P aec52eb99aef4416b5d7c9a64ec945d3: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:03.134953 8997 log.cc:824] T 33f1929e336e4938a7ac0d19dea72055 P aec52eb99aef4416b5d7c9a64ec945d3: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:03.135563 8998 tablet_bootstrap.cc:492] T 33f1929e336e4938a7ac0d19dea72055 P 672f7b7ab39543f99858fcd65804246b: No bootstrap required, opened a new log
I20260619 20:32:03.135639 8998 ts_tablet_manager.cc:1397] T 33f1929e336e4938a7ac0d19dea72055 P 672f7b7ab39543f99858fcd65804246b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 20:32:03.135670 8997 tablet_bootstrap.cc:492] T 33f1929e336e4938a7ac0d19dea72055 P aec52eb99aef4416b5d7c9a64ec945d3: No bootstrap required, opened a new log
I20260619 20:32:03.135772 8997 ts_tablet_manager.cc:1397] T 33f1929e336e4938a7ac0d19dea72055 P aec52eb99aef4416b5d7c9a64ec945d3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:32:03.136463 8996 raft_consensus.cc:348] T 33f1929e336e4938a7ac0d19dea72055 P faacb878616b418b90a54723a418ecee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aec52eb99aef4416b5d7c9a64ec945d3" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45185 } } peers { permanent_uuid: "672f7b7ab39543f99858fcd65804246b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37401 } } peers { permanent_uuid: "faacb878616b418b90a54723a418ecee" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 43437 } }
I20260619 20:32:03.136763 8996 raft_consensus.cc:374] T 33f1929e336e4938a7ac0d19dea72055 P faacb878616b418b90a54723a418ecee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:03.136824 8996 raft_consensus.cc:729] T 33f1929e336e4938a7ac0d19dea72055 P faacb878616b418b90a54723a418ecee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faacb878616b418b90a54723a418ecee, State: Initialized, Role: FOLLOWER
I20260619 20:32:03.136974 8996 consensus_queue.cc:260] T 33f1929e336e4938a7ac0d19dea72055 P faacb878616b418b90a54723a418ecee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aec52eb99aef4416b5d7c9a64ec945d3" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45185 } } peers { permanent_uuid: "672f7b7ab39543f99858fcd65804246b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37401 } } peers { permanent_uuid: "faacb878616b418b90a54723a418ecee" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 43437 } }
I20260619 20:32:03.137310 8996 ts_tablet_manager.cc:1428] T 33f1929e336e4938a7ac0d19dea72055 P faacb878616b418b90a54723a418ecee: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:32:03.137444 8846 heartbeater.cc:503] Master 127.4.83.126:37801 was elected leader, sending a full tablet report...
I20260619 20:32:03.137867 8998 raft_consensus.cc:348] T 33f1929e336e4938a7ac0d19dea72055 P 672f7b7ab39543f99858fcd65804246b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aec52eb99aef4416b5d7c9a64ec945d3" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45185 } } peers { permanent_uuid: "672f7b7ab39543f99858fcd65804246b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37401 } } peers { permanent_uuid: "faacb878616b418b90a54723a418ecee" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 43437 } }
I20260619 20:32:03.138003 8998 raft_consensus.cc:374] T 33f1929e336e4938a7ac0d19dea72055 P 672f7b7ab39543f99858fcd65804246b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:03.138033 8998 raft_consensus.cc:729] T 33f1929e336e4938a7ac0d19dea72055 P 672f7b7ab39543f99858fcd65804246b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 672f7b7ab39543f99858fcd65804246b, State: Initialized, Role: FOLLOWER
I20260619 20:32:03.137943 8997 raft_consensus.cc:348] T 33f1929e336e4938a7ac0d19dea72055 P aec52eb99aef4416b5d7c9a64ec945d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aec52eb99aef4416b5d7c9a64ec945d3" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45185 } } peers { permanent_uuid: "672f7b7ab39543f99858fcd65804246b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37401 } } peers { permanent_uuid: "faacb878616b418b90a54723a418ecee" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 43437 } }
I20260619 20:32:03.138159 8997 raft_consensus.cc:374] T 33f1929e336e4938a7ac0d19dea72055 P aec52eb99aef4416b5d7c9a64ec945d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:03.138211 8997 raft_consensus.cc:729] T 33f1929e336e4938a7ac0d19dea72055 P aec52eb99aef4416b5d7c9a64ec945d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aec52eb99aef4416b5d7c9a64ec945d3, State: Initialized, Role: FOLLOWER
I20260619 20:32:03.138181 8998 consensus_queue.cc:260] T 33f1929e336e4938a7ac0d19dea72055 P 672f7b7ab39543f99858fcd65804246b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aec52eb99aef4416b5d7c9a64ec945d3" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45185 } } peers { permanent_uuid: "672f7b7ab39543f99858fcd65804246b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37401 } } peers { permanent_uuid: "faacb878616b418b90a54723a418ecee" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 43437 } }
I20260619 20:32:03.138378 8997 consensus_queue.cc:260] T 33f1929e336e4938a7ac0d19dea72055 P aec52eb99aef4416b5d7c9a64ec945d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aec52eb99aef4416b5d7c9a64ec945d3" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45185 } } peers { permanent_uuid: "672f7b7ab39543f99858fcd65804246b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37401 } } peers { permanent_uuid: "faacb878616b418b90a54723a418ecee" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 43437 } }
I20260619 20:32:03.138463 8998 ts_tablet_manager.cc:1428] T 33f1929e336e4938a7ac0d19dea72055 P 672f7b7ab39543f99858fcd65804246b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:32:03.138643 8979 heartbeater.cc:503] Master 127.4.83.126:37801 was elected leader, sending a full tablet report...
I20260619 20:32:03.138720 8997 ts_tablet_manager.cc:1428] T 33f1929e336e4938a7ac0d19dea72055 P aec52eb99aef4416b5d7c9a64ec945d3: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 20:32:03.138794 8713 heartbeater.cc:503] Master 127.4.83.126:37801 was elected leader, sending a full tablet report...
W20260619 20:32:03.222232 8714 tablet.cc:2369] T 33f1929e336e4938a7ac0d19dea72055 P aec52eb99aef4416b5d7c9a64ec945d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:32:03.280119 9004 raft_consensus.cc:482] T 33f1929e336e4938a7ac0d19dea72055 P aec52eb99aef4416b5d7c9a64ec945d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:32:03.280282 9004 raft_consensus.cc:504] T 33f1929e336e4938a7ac0d19dea72055 P aec52eb99aef4416b5d7c9a64ec945d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aec52eb99aef4416b5d7c9a64ec945d3" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45185 } } peers { permanent_uuid: "672f7b7ab39543f99858fcd65804246b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37401 } } peers { permanent_uuid: "faacb878616b418b90a54723a418ecee" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 43437 } }
I20260619 20:32:03.280633 9004 leader_election.cc:288] T 33f1929e336e4938a7ac0d19dea72055 P aec52eb99aef4416b5d7c9a64ec945d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 672f7b7ab39543f99858fcd65804246b (127.4.83.67:37401), faacb878616b418b90a54723a418ecee (127.4.83.66:43437)
I20260619 20:32:03.284953 8993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:03.280956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:39951 (local address 127.4.83.67:37401)
0619 20:32:03.281037 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:03.281039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:03.281082 (+ 43us) server_negotiation.cc:407] Connection header received
0619 20:32:03.281112 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:03.281112 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:03.281122 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:03.281174 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:03.281338 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:03.282086 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:03.283689 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:03.284449 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:03.284541 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:03.284554 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:03.284846 (+ 292us) server_negotiation.cc:299] Negotiation successful
0619 20:32:03.284894 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 20:32:03.284935 8995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:03.281109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:38179 (local address 127.4.83.66:43437)
0619 20:32:03.281172 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:03.281174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:03.281182 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:32:03.281208 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:03.281209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:03.281221 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:03.281265 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:03.281398 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:03.282107 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:03.283689 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:03.284449 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:03.284541 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:03.284554 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:03.284813 (+ 259us) server_negotiation.cc:299] Negotiation successful
0619 20:32:03.284864 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 20:32:03.284961 9005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:03.280830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:37401 (local address 127.4.83.65:39951)
0619 20:32:03.280948 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:03.280960 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:03.281016 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:03.281191 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:03.281192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:03.281194 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:03.281272 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:03.281273 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:03.282216 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:03.282218 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:03.283555 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:03.283560 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:03.284639 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:03.284640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:03.284714 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:03.284716 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:32:03.284779 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 20:32:03.284841 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":76,"thread_start_us":68,"threads_started":1}
I20260619 20:32:03.284966 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:03.281005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:43437 (local address 127.4.83.65:38179)
0619 20:32:03.281107 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:03.281115 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:03.281155 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:03.281276 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:03.281277 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:03.281278 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:03.281337 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:03.281338 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:03.282216 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:03.282218 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:03.283555 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:03.283560 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:03.284573 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:03.284575 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:03.284708 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:03.284714 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 20:32:03.284779 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 20:32:03.284840 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":41,"thread_start_us":50,"threads_started":1}
I20260619 20:32:03.285413 8933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33f1929e336e4938a7ac0d19dea72055" candidate_uuid: "aec52eb99aef4416b5d7c9a64ec945d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "672f7b7ab39543f99858fcd65804246b" is_pre_election: true
I20260619 20:32:03.285573 8933 raft_consensus.cc:2454] T 33f1929e336e4938a7ac0d19dea72055 P 672f7b7ab39543f99858fcd65804246b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aec52eb99aef4416b5d7c9a64ec945d3 in term 0.
I20260619 20:32:03.285579 8800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33f1929e336e4938a7ac0d19dea72055" candidate_uuid: "aec52eb99aef4416b5d7c9a64ec945d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faacb878616b418b90a54723a418ecee" is_pre_election: true
I20260619 20:32:03.285735 8800 raft_consensus.cc:2454] T 33f1929e336e4938a7ac0d19dea72055 P faacb878616b418b90a54723a418ecee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aec52eb99aef4416b5d7c9a64ec945d3 in term 0.
I20260619 20:32:03.285791 8602 leader_election.cc:302] T 33f1929e336e4938a7ac0d19dea72055 P aec52eb99aef4416b5d7c9a64ec945d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 672f7b7ab39543f99858fcd65804246b, aec52eb99aef4416b5d7c9a64ec945d3; no voters:
I20260619 20:32:03.285923 9004 raft_consensus.cc:2781] T 33f1929e336e4938a7ac0d19dea72055 P aec52eb99aef4416b5d7c9a64ec945d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:32:03.285982 9004 raft_consensus.cc:482] T 33f1929e336e4938a7ac0d19dea72055 P aec52eb99aef4416b5d7c9a64ec945d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:32:03.286000 9004 raft_consensus.cc:3037] T 33f1929e336e4938a7ac0d19dea72055 P aec52eb99aef4416b5d7c9a64ec945d3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:03.286715 9004 raft_consensus.cc:504] T 33f1929e336e4938a7ac0d19dea72055 P aec52eb99aef4416b5d7c9a64ec945d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aec52eb99aef4416b5d7c9a64ec945d3" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45185 } } peers { permanent_uuid: "672f7b7ab39543f99858fcd65804246b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37401 } } peers { permanent_uuid: "faacb878616b418b90a54723a418ecee" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 43437 } }
I20260619 20:32:03.286850 9004 leader_election.cc:288] T 33f1929e336e4938a7ac0d19dea72055 P aec52eb99aef4416b5d7c9a64ec945d3 [CANDIDATE]: Term 1 election: Requested vote from peers 672f7b7ab39543f99858fcd65804246b (127.4.83.67:37401), faacb878616b418b90a54723a418ecee (127.4.83.66:43437)
I20260619 20:32:03.287056 8933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33f1929e336e4938a7ac0d19dea72055" candidate_uuid: "aec52eb99aef4416b5d7c9a64ec945d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "672f7b7ab39543f99858fcd65804246b"
I20260619 20:32:03.287129 8800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33f1929e336e4938a7ac0d19dea72055" candidate_uuid: "aec52eb99aef4416b5d7c9a64ec945d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faacb878616b418b90a54723a418ecee"
I20260619 20:32:03.287191 8800 raft_consensus.cc:3037] T 33f1929e336e4938a7ac0d19dea72055 P faacb878616b418b90a54723a418ecee [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:03.287189 8933 raft_consensus.cc:3037] T 33f1929e336e4938a7ac0d19dea72055 P 672f7b7ab39543f99858fcd65804246b [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:03.287968 8933 raft_consensus.cc:2454] T 33f1929e336e4938a7ac0d19dea72055 P 672f7b7ab39543f99858fcd65804246b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aec52eb99aef4416b5d7c9a64ec945d3 in term 1.
I20260619 20:32:03.288017 8800 raft_consensus.cc:2454] T 33f1929e336e4938a7ac0d19dea72055 P faacb878616b418b90a54723a418ecee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aec52eb99aef4416b5d7c9a64ec945d3 in term 1.
I20260619 20:32:03.288185 8602 leader_election.cc:302] T 33f1929e336e4938a7ac0d19dea72055 P aec52eb99aef4416b5d7c9a64ec945d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aec52eb99aef4416b5d7c9a64ec945d3, faacb878616b418b90a54723a418ecee; no voters:
I20260619 20:32:03.288295 9004 raft_consensus.cc:2781] T 33f1929e336e4938a7ac0d19dea72055 P aec52eb99aef4416b5d7c9a64ec945d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:32:03.288439 9004 raft_consensus.cc:686] T 33f1929e336e4938a7ac0d19dea72055 P aec52eb99aef4416b5d7c9a64ec945d3 [term 1 LEADER]: Becoming Leader. State: Replica: aec52eb99aef4416b5d7c9a64ec945d3, State: Running, Role: LEADER
I20260619 20:32:03.288523 9004 consensus_queue.cc:237] T 33f1929e336e4938a7ac0d19dea72055 P aec52eb99aef4416b5d7c9a64ec945d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aec52eb99aef4416b5d7c9a64ec945d3" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45185 } } peers { permanent_uuid: "672f7b7ab39543f99858fcd65804246b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37401 } } peers { permanent_uuid: "faacb878616b418b90a54723a418ecee" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 43437 } }
I20260619 20:32:03.289467 8522 catalog_manager.cc:5495] T 33f1929e336e4938a7ac0d19dea72055 P aec52eb99aef4416b5d7c9a64ec945d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to aec52eb99aef4416b5d7c9a64ec945d3 (127.4.83.65). New cstate: current_term: 1 leader_uuid: "aec52eb99aef4416b5d7c9a64ec945d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aec52eb99aef4416b5d7c9a64ec945d3" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "672f7b7ab39543f99858fcd65804246b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "faacb878616b418b90a54723a418ecee" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 43437 } health_report { overall_health: UNKNOWN } } }
I20260619 20:32:03.316079 8992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:03.312657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52962 (local address 127.4.83.65:45185)
0619 20:32:03.312723 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:03.312725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:03.312733 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:32:03.312768 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:03.312769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:03.312789 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:03.312848 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:32:03.313076 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:03.313606 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:03.314696 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:03.314895 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:03.314957 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:03.315209 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:32:03.315323 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:32:03.315577 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:03.315594 (+ 17us) server_negotiation.cc:299] Negotiation successful
0619 20:32:03.315750 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 20:32:03.318755 8933 raft_consensus.cc:1261] T 33f1929e336e4938a7ac0d19dea72055 P 672f7b7ab39543f99858fcd65804246b [term 1 FOLLOWER]: Refusing update from remote peer aec52eb99aef4416b5d7c9a64ec945d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:32:03.319005 8800 raft_consensus.cc:1261] T 33f1929e336e4938a7ac0d19dea72055 P faacb878616b418b90a54723a418ecee [term 1 FOLLOWER]: Refusing update from remote peer aec52eb99aef4416b5d7c9a64ec945d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:32:03.319123 9007 consensus_queue.cc:1035] T 33f1929e336e4938a7ac0d19dea72055 P aec52eb99aef4416b5d7c9a64ec945d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "672f7b7ab39543f99858fcd65804246b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:32:03.319334 9007 consensus_queue.cc:1035] T 33f1929e336e4938a7ac0d19dea72055 P aec52eb99aef4416b5d7c9a64ec945d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "faacb878616b418b90a54723a418ecee" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 43437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:32:03.321874 9011 mvcc.cc:205] Tried to move back new op lower bound from 7298667001110626304 to 7298667000990425088. Current Snapshot: MvccSnapshot[applied={T|T < 7298667001110626304}]
I20260619 20:32:03.322827 9012 mvcc.cc:205] Tried to move back new op lower bound from 7298667001110626304 to 7298667000990425088. Current Snapshot: MvccSnapshot[applied={T|T < 7298667001110626304}]
I20260619 20:32:03.324368 8522 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:32:03.324469 8522 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:32:03.325373 8522 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ac256e9a1e8d47ac984cb483569db7d8: Sending DeleteTablet for 3 replicas of tablet 33f1929e336e4938a7ac0d19dea72055
I20260619 20:32:03.325739 8780 tablet_service.cc:1514] Processing DeleteTablet for tablet 33f1929e336e4938a7ac0d19dea72055 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:32:03 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:44210
I20260619 20:32:03.325807 8912 tablet_service.cc:1514] Processing DeleteTablet for tablet 33f1929e336e4938a7ac0d19dea72055 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:32:03 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:54674
I20260619 20:32:03.326002 8644 tablet_service.cc:1514] Processing DeleteTablet for tablet 33f1929e336e4938a7ac0d19dea72055 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:32:03 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:52948
I20260619 20:32:03.326238 9022 tablet_replica.cc:330] T 33f1929e336e4938a7ac0d19dea72055 P aec52eb99aef4416b5d7c9a64ec945d3: stopping tablet replica
I20260619 20:32:03.326283 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 8585
I20260619 20:32:03.327168 9020 tablet_replica.cc:330] T 33f1929e336e4938a7ac0d19dea72055 P faacb878616b418b90a54723a418ecee: stopping tablet replica
I20260619 20:32:03.327386 9020 raft_consensus.cc:2229] T 33f1929e336e4938a7ac0d19dea72055 P faacb878616b418b90a54723a418ecee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:32:03.327541 9020 raft_consensus.cc:2258] T 33f1929e336e4938a7ac0d19dea72055 P faacb878616b418b90a54723a418ecee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:32:03.328024 9021 tablet_replica.cc:330] T 33f1929e336e4938a7ac0d19dea72055 P 672f7b7ab39543f99858fcd65804246b: stopping tablet replica
I20260619 20:32:03.328131 9020 ts_tablet_manager.cc:1905] T 33f1929e336e4938a7ac0d19dea72055 P faacb878616b418b90a54723a418ecee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:32:03.328243 9021 raft_consensus.cc:2229] T 33f1929e336e4938a7ac0d19dea72055 P 672f7b7ab39543f99858fcd65804246b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:32:03.328410 9021 raft_consensus.cc:2258] T 33f1929e336e4938a7ac0d19dea72055 P 672f7b7ab39543f99858fcd65804246b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:32:03.328913 9021 ts_tablet_manager.cc:1905] T 33f1929e336e4938a7ac0d19dea72055 P 672f7b7ab39543f99858fcd65804246b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:32:03.330225 9020 ts_tablet_manager.cc:1918] T 33f1929e336e4938a7ac0d19dea72055 P faacb878616b418b90a54723a418ecee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:32:03.330317 9020 log.cc:1196] T 33f1929e336e4938a7ac0d19dea72055 P faacb878616b418b90a54723a418ecee: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-1/wal/wals/33f1929e336e4938a7ac0d19dea72055
I20260619 20:32:03.330670 9020 ts_tablet_manager.cc:1939] T 33f1929e336e4938a7ac0d19dea72055 P faacb878616b418b90a54723a418ecee: Deleting consensus metadata
I20260619 20:32:03.330899 9021 ts_tablet_manager.cc:1918] T 33f1929e336e4938a7ac0d19dea72055 P 672f7b7ab39543f99858fcd65804246b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:32:03.330981 9021 log.cc:1196] T 33f1929e336e4938a7ac0d19dea72055 P 672f7b7ab39543f99858fcd65804246b: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901102012824-4429-0/minicluster-data/ts-2/wal/wals/33f1929e336e4938a7ac0d19dea72055
I20260619 20:32:03.331143 8509 catalog_manager.cc:4841] TS faacb878616b418b90a54723a418ecee (127.4.83.66:43437): tablet 33f1929e336e4938a7ac0d19dea72055 (table test-table [id=64aab6e10cdd412cae5bdf64f3ed8baa]) successfully deleted
I20260619 20:32:03.331244 9021 ts_tablet_manager.cc:1939] T 33f1929e336e4938a7ac0d19dea72055 P 672f7b7ab39543f99858fcd65804246b: Deleting consensus metadata
I20260619 20:32:03.331686 8509 catalog_manager.cc:4841] TS 672f7b7ab39543f99858fcd65804246b (127.4.83.67:37401): tablet 33f1929e336e4938a7ac0d19dea72055 (table test-table [id=64aab6e10cdd412cae5bdf64f3ed8baa]) successfully deleted
W20260619 20:32:03.332561 8507 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.83.65:45185 (error 108)
I20260619 20:32:03.332757 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 8718
I20260619 20:32:03.336824 8991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:03.336581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:45185 (local address 127.0.0.1:52966)
0619 20:32:03.336703 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:03.336779 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.65:45185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260619 20:32:03.336993 8507 catalog_manager.cc:4584] TS aec52eb99aef4416b5d7c9a64ec945d3 (127.4.83.65:45185): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 33f1929e336e4938a7ac0d19dea72055: Network error: Client connection negotiation failed: client connection to 127.4.83.65:45185: connect: Connection refused (error 111)
I20260619 20:32:03.337256 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 8851
I20260619 20:32:03.344177 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 8492
2026-06-19T20:32:03Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2061 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-19T20:32:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:32:03Z Disabled control of system clock
I20260619 20:32:03.371645 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:45617
--webserver_interface=127.4.83.126
--webserver_port=0
--builtin_ntp_servers=127.4.83.84:35061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:45617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 20:32:03.453691 9027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:03.453915 9027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:03.453939 9027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:03.455163 9027 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:32:03.455216 9027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:03.455227 9027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:03.455240 9027 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:32:03.455250 9027 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:32:03.456713 9027 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:35061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:45617
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:45617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:03.456982 9027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:03.457413 9027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:03.463302 9035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:03.463209 9032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:03.463229 9033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:03.463474 9027 server_base.cc:1034] running on GCE node
I20260619 20:32:03.463878 9027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:03.464206 9027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:03.465363 9027 hybrid_clock.cc:648] HybridClock initialized: now 1781901123465354 us; error 32 us; skew 500 ppm
I20260619 20:32:03.466912 9027 webserver.cc:460] Webserver started at http://127.4.83.126:33425/ using document root <none> and password file <none>
I20260619 20:32:03.467175 9027 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:03.467224 9027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:03.467361 9027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:03.468698 9027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "084e302719144ffa99ebfcda8b867dee"
format_stamp: "Formatted at 2026-06-19 20:32:03 on dist-test-slave-4m58"
server_key: "d895f3bee02458006d54ef43c3f20802"
server_key_iv: "646d32484b1f01d2adffb2797d8e110a"
server_key_version: "encryptionkey@0"
I20260619 20:32:03.469065 9027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "084e302719144ffa99ebfcda8b867dee"
format_stamp: "Formatted at 2026-06-19 20:32:03 on dist-test-slave-4m58"
server_key: "d895f3bee02458006d54ef43c3f20802"
server_key_iv: "646d32484b1f01d2adffb2797d8e110a"
server_key_version: "encryptionkey@0"
I20260619 20:32:03.470504 9027 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:32:03.471511 9041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:03.471735 9027 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:03.471854 9027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "084e302719144ffa99ebfcda8b867dee"
format_stamp: "Formatted at 2026-06-19 20:32:03 on dist-test-slave-4m58"
server_key: "d895f3bee02458006d54ef43c3f20802"
server_key_iv: "646d32484b1f01d2adffb2797d8e110a"
server_key_version: "encryptionkey@0"
I20260619 20:32:03.471936 9027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:03.475481 9027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:03.475771 9027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:03.476059 9027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:03.480975 9027 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:45617
I20260619 20:32:03.481037 9093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:45617 every 8 connection(s)
I20260619 20:32:03.481385 9027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:32:03.482096 9094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:03.485323 9094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 084e302719144ffa99ebfcda8b867dee: Bootstrap starting.
I20260619 20:32:03.485960 9094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 084e302719144ffa99ebfcda8b867dee: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:03.486421 9094 log.cc:824] T 00000000000000000000000000000000 P 084e302719144ffa99ebfcda8b867dee: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:03.486434 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 9027
I20260619 20:32:03.486529 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:32:03.486716 4429 external_mini_cluster.cc:1428] Setting key f2bfd994ca0e722a477ec569e9d82228
I20260619 20:32:03.487339 9094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 084e302719144ffa99ebfcda8b867dee: No bootstrap required, opened a new log
I20260619 20:32:03.489992 9094 raft_consensus.cc:348] T 00000000000000000000000000000000 P 084e302719144ffa99ebfcda8b867dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "084e302719144ffa99ebfcda8b867dee" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45617 } }
I20260619 20:32:03.490214 9094 raft_consensus.cc:374] T 00000000000000000000000000000000 P 084e302719144ffa99ebfcda8b867dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:03.490257 9094 raft_consensus.cc:729] T 00000000000000000000000000000000 P 084e302719144ffa99ebfcda8b867dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 084e302719144ffa99ebfcda8b867dee, State: Initialized, Role: FOLLOWER
I20260619 20:32:03.490388 9094 consensus_queue.cc:260] T 00000000000000000000000000000000 P 084e302719144ffa99ebfcda8b867dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "084e302719144ffa99ebfcda8b867dee" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45617 } }
I20260619 20:32:03.490470 9098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:03.487328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54122 (local address 127.4.83.126:45617)
0619 20:32:03.487597 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:03.487607 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:03.487641 (+ 34us) server_negotiation.cc:407] Connection header received
0619 20:32:03.488926 (+ 1285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:03.488936 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:03.488995 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:03.489301 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:03.489955 (+ 654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:03.489958 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:03.489959 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:32:03.489963 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:03.490012 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:32:03.490027 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:03.490114 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:03.490216 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 20:32:03.490281 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":104,"threads_started":1}
I20260619 20:32:03.490481 9094 raft_consensus.cc:388] T 00000000000000000000000000000000 P 084e302719144ffa99ebfcda8b867dee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:32:03.490553 9094 raft_consensus.cc:482] T 00000000000000000000000000000000 P 084e302719144ffa99ebfcda8b867dee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:32:03.490658 9094 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 084e302719144ffa99ebfcda8b867dee [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:03.491815 9094 raft_consensus.cc:504] T 00000000000000000000000000000000 P 084e302719144ffa99ebfcda8b867dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "084e302719144ffa99ebfcda8b867dee" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45617 } }
I20260619 20:32:03.491971 9094 leader_election.cc:302] T 00000000000000000000000000000000 P 084e302719144ffa99ebfcda8b867dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 084e302719144ffa99ebfcda8b867dee; no voters:
I20260619 20:32:03.492216 9094 leader_election.cc:288] T 00000000000000000000000000000000 P 084e302719144ffa99ebfcda8b867dee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:32:03.492274 9099 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 084e302719144ffa99ebfcda8b867dee [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:32:03.492408 9099 raft_consensus.cc:686] T 00000000000000000000000000000000 P 084e302719144ffa99ebfcda8b867dee [term 1 LEADER]: Becoming Leader. State: Replica: 084e302719144ffa99ebfcda8b867dee, State: Running, Role: LEADER
I20260619 20:32:03.492532 9094 sys_catalog.cc:564] T 00000000000000000000000000000000 P 084e302719144ffa99ebfcda8b867dee [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:32:03.492547 9099 consensus_queue.cc:237] T 00000000000000000000000000000000 P 084e302719144ffa99ebfcda8b867dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "084e302719144ffa99ebfcda8b867dee" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45617 } }
I20260619 20:32:03.492988 9100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 084e302719144ffa99ebfcda8b867dee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "084e302719144ffa99ebfcda8b867dee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "084e302719144ffa99ebfcda8b867dee" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45617 } } }
I20260619 20:32:03.492987 9101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 084e302719144ffa99ebfcda8b867dee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 084e302719144ffa99ebfcda8b867dee. Latest consensus state: current_term: 1 leader_uuid: "084e302719144ffa99ebfcda8b867dee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "084e302719144ffa99ebfcda8b867dee" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45617 } } }
I20260619 20:32:03.493106 9100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 084e302719144ffa99ebfcda8b867dee [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:03.493110 9101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 084e302719144ffa99ebfcda8b867dee [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:03.493335 9106 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:32:03.493809 9106 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:32:03.495435 9106 catalog_manager.cc:1347] Generated new cluster ID: 50d593c1571c45359ae79cd0408ba14d
I20260619 20:32:03.495501 9106 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:32:03.513620 9106 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:32:03.513813 9106 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:32:03.517150 9106 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 084e302719144ffa99ebfcda8b867dee: Generated new TSK 0
I20260619 20:32:03.517333 9106 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:32:03.522149 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:0
--local_ip_for_outbound_sockets=127.4.83.65
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:45617
--builtin_ntp_servers=127.4.83.84:35061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 20:32:03.605785 9118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:03.606035 9118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:03.606060 9118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:03.607630 9118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:03.607695 9118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:03.607735 9118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:32:03.609573 9118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:35061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:45617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:03.609879 9118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:03.610201 9118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:03.616094 9123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:03.616106 9124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:03.616096 9126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:03.616463 9118 server_base.cc:1034] running on GCE node
I20260619 20:32:03.616762 9118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:03.617038 9118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:03.618203 9118 hybrid_clock.cc:648] HybridClock initialized: now 1781901123618173 us; error 49 us; skew 500 ppm
I20260619 20:32:03.619560 9118 webserver.cc:460] Webserver started at http://127.4.83.65:39543/ using document root <none> and password file <none>
I20260619 20:32:03.619796 9118 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:03.619875 9118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:03.620010 9118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:03.621124 9118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-0/data/instance:
uuid: "a7b9828576d64060bbbb696f322ff308"
format_stamp: "Formatted at 2026-06-19 20:32:03 on dist-test-slave-4m58"
server_key: "35e7dec809b56b0b2143e8b6ad221dad"
server_key_iv: "55bece359ac7cf03062cc30db3b13fb4"
server_key_version: "encryptionkey@0"
I20260619 20:32:03.621476 9118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance:
uuid: "a7b9828576d64060bbbb696f322ff308"
format_stamp: "Formatted at 2026-06-19 20:32:03 on dist-test-slave-4m58"
server_key: "35e7dec809b56b0b2143e8b6ad221dad"
server_key_iv: "55bece359ac7cf03062cc30db3b13fb4"
server_key_version: "encryptionkey@0"
I20260619 20:32:03.622849 9118 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:32:03.623606 9132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:03.623782 9118 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:32:03.623903 9118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "a7b9828576d64060bbbb696f322ff308"
format_stamp: "Formatted at 2026-06-19 20:32:03 on dist-test-slave-4m58"
server_key: "35e7dec809b56b0b2143e8b6ad221dad"
server_key_iv: "55bece359ac7cf03062cc30db3b13fb4"
server_key_version: "encryptionkey@0"
I20260619 20:32:03.623982 9118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:03.627736 9118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:03.628026 9118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:03.628153 9118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:03.628357 9118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:03.628667 9118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:03.628701 9118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:03.628731 9118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:03.628755 9118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:03.634260 9118 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:39541
I20260619 20:32:03.634296 9245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:39541 every 8 connection(s)
I20260619 20:32:03.634634 9118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
I20260619 20:32:03.637156 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 9118
I20260619 20:32:03.637286 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance
I20260619 20:32:03.637477 4429 external_mini_cluster.cc:1428] Setting key 1fcdf4e2239f41210b69c29c87083787
I20260619 20:32:03.637950 9098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:03.635315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:59231 (local address 127.4.83.126:45617)
0619 20:32:03.635360 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:03.635362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:03.635574 (+ 212us) server_negotiation.cc:407] Connection header received
0619 20:32:03.636257 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:03.636259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:03.636267 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:03.636319 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:03.637584 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:03.637585 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:03.637585 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:32:03.637589 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:03.637606 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:32:03.637610 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:03.637748 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:03.637866 (+ 118us) server_negotiation.cc:299] Negotiation successful
0619 20:32:03.637883 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:32:03.638145 9248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:03.635290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:45617 (local address 127.4.83.65:59231)
0619 20:32:03.635463 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:03.635497 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:03.636105 (+ 608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:03.636402 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:03.636406 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:03.636416 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:03.637407 (+ 991us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:32:03.637407 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:03.637448 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:32:03.637481 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:03.637750 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:03.637757 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:03.637761 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:32:03.637842 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 20:32:03.637903 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":81,"threads_started":1}
I20260619 20:32:03.638491 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:0
--local_ip_for_outbound_sockets=127.4.83.66
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:45617
--builtin_ntp_servers=127.4.83.84:35061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260619 20:32:03.639752 9246 heartbeater.cc:347] Connected to a master server at 127.4.83.126:45617
I20260619 20:32:03.639882 9246 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:03.640035 9246 heartbeater.cc:511] Master 127.4.83.126:45617 requested a full tablet report, sending...
I20260619 20:32:03.640563 9058 ts_manager.cc:194] Registered new tserver with Master: a7b9828576d64060bbbb696f322ff308 (127.4.83.65:39541)
W20260619 20:32:03.717900 9249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:03.718142 9249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:03.718176 9249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:03.719597 9249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:03.719656 9249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:03.719692 9249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:32:03.721186 9249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:35061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:45617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:03.721390 9249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:03.721647 9249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:03.726643 9254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:03.726639 9257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:03.726643 9255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:03.726893 9249 server_base.cc:1034] running on GCE node
I20260619 20:32:03.727195 9249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:03.727456 9249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:03.728637 9249 hybrid_clock.cc:648] HybridClock initialized: now 1781901123728616 us; error 37 us; skew 500 ppm
I20260619 20:32:03.729934 9249 webserver.cc:460] Webserver started at http://127.4.83.66:46541/ using document root <none> and password file <none>
I20260619 20:32:03.730158 9249 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:03.730208 9249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:03.730340 9249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:03.731470 9249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-1/data/instance:
uuid: "a6c00d1d0ed748f2a4734f96a8fce485"
format_stamp: "Formatted at 2026-06-19 20:32:03 on dist-test-slave-4m58"
server_key: "67aba6b808efe13fa4077331689a6fe7"
server_key_iv: "e18945295293c5f5bc1f1983227f462c"
server_key_version: "encryptionkey@0"
I20260619 20:32:03.731806 9249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance:
uuid: "a6c00d1d0ed748f2a4734f96a8fce485"
format_stamp: "Formatted at 2026-06-19 20:32:03 on dist-test-slave-4m58"
server_key: "67aba6b808efe13fa4077331689a6fe7"
server_key_iv: "e18945295293c5f5bc1f1983227f462c"
server_key_version: "encryptionkey@0"
I20260619 20:32:03.733026 9249 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:03.733747 9263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:03.733888 9249 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:32:03.733954 9249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "a6c00d1d0ed748f2a4734f96a8fce485"
format_stamp: "Formatted at 2026-06-19 20:32:03 on dist-test-slave-4m58"
server_key: "67aba6b808efe13fa4077331689a6fe7"
server_key_iv: "e18945295293c5f5bc1f1983227f462c"
server_key_version: "encryptionkey@0"
I20260619 20:32:03.734048 9249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:03.737294 9249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:03.737519 9249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:03.737627 9249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:03.737797 9249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:03.738061 9249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:03.738101 9249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:03.738128 9249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:03.738150 9249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:03.743474 9249 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:39085
I20260619 20:32:03.743528 9376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:39085 every 8 connection(s)
I20260619 20:32:03.743853 9249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
I20260619 20:32:03.746940 9098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:03.744603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:37895 (local address 127.4.83.126:45617)
0619 20:32:03.744722 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:03.744725 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:03.744913 (+ 188us) server_negotiation.cc:407] Connection header received
0619 20:32:03.745440 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:03.745442 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:03.745448 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:03.745497 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:03.746694 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:03.746696 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:03.746696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:32:03.746701 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:03.746717 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:32:03.746722 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:03.746773 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:03.746846 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 20:32:03.746861 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260619 20:32:03.747131 9379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:03.744538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:45617 (local address 127.4.83.66:37895)
0619 20:32:03.744792 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:03.744828 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:03.745322 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:03.745580 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:03.745585 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:03.745595 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:03.746538 (+ 943us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:32:03.746539 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:03.746570 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:32:03.746599 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:03.746776 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:03.746781 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:03.746785 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:32:03.746854 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 20:32:03.746919 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":90,"threads_started":1}
I20260619 20:32:03.747668 9377 heartbeater.cc:347] Connected to a master server at 127.4.83.126:45617
I20260619 20:32:03.747758 9377 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:03.747931 9377 heartbeater.cc:511] Master 127.4.83.126:45617 requested a full tablet report, sending...
I20260619 20:32:03.748339 9058 ts_manager.cc:194] Registered new tserver with Master: a6c00d1d0ed748f2a4734f96a8fce485 (127.4.83.66:39085)
I20260619 20:32:03.753276 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 9249
I20260619 20:32:03.753364 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance
I20260619 20:32:03.753509 4429 external_mini_cluster.cc:1428] Setting key 4d818c9222c5cb158e2d591b42b045cd
I20260619 20:32:03.754429 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:0
--local_ip_for_outbound_sockets=127.4.83.67
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:45617
--builtin_ntp_servers=127.4.83.84:35061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 20:32:03.835430 9380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:03.835669 9380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:03.835695 9380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:03.837289 9380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:03.837349 9380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:03.837380 9380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:32:03.838896 9380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:35061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:45617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:03.839143 9380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:03.839414 9380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:03.844571 9385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:03.844594 9386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:03.844647 9380 server_base.cc:1034] running on GCE node
W20260619 20:32:03.844583 9388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:03.844954 9380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:03.845216 9380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:03.846386 9380 hybrid_clock.cc:648] HybridClock initialized: now 1781901123846374 us; error 31 us; skew 500 ppm
I20260619 20:32:03.848012 9380 webserver.cc:460] Webserver started at http://127.4.83.67:35029/ using document root <none> and password file <none>
I20260619 20:32:03.848266 9380 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:03.848318 9380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:03.848448 9380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:03.849529 9380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-2/data/instance:
uuid: "1f06008a99d14b069a6c97f8b2731d43"
format_stamp: "Formatted at 2026-06-19 20:32:03 on dist-test-slave-4m58"
server_key: "02cabee925c93f02e6020ff953905c32"
server_key_iv: "44c69b4f3f00ab0f5ea0b97f794c71d4"
server_key_version: "encryptionkey@0"
I20260619 20:32:03.849862 9380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance:
uuid: "1f06008a99d14b069a6c97f8b2731d43"
format_stamp: "Formatted at 2026-06-19 20:32:03 on dist-test-slave-4m58"
server_key: "02cabee925c93f02e6020ff953905c32"
server_key_iv: "44c69b4f3f00ab0f5ea0b97f794c71d4"
server_key_version: "encryptionkey@0"
I20260619 20:32:03.851143 9380 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:32:03.851907 9394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:03.852061 9380 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:32:03.852154 9380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "1f06008a99d14b069a6c97f8b2731d43"
format_stamp: "Formatted at 2026-06-19 20:32:03 on dist-test-slave-4m58"
server_key: "02cabee925c93f02e6020ff953905c32"
server_key_iv: "44c69b4f3f00ab0f5ea0b97f794c71d4"
server_key_version: "encryptionkey@0"
I20260619 20:32:03.852216 9380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:03.856256 9380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:03.856552 9380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:03.856710 9380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:03.856918 9380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:03.857261 9380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:03.857304 9380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:03.857340 9380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:03.857358 9380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:03.863346 9380 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:33547
I20260619 20:32:03.863399 9507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:33547 every 8 connection(s)
I20260619 20:32:03.863767 9380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
I20260619 20:32:03.866778 9098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:03.864501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:36741 (local address 127.4.83.126:45617)
0619 20:32:03.864560 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:03.864562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:03.864783 (+ 221us) server_negotiation.cc:407] Connection header received
0619 20:32:03.865299 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:03.865300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:03.865308 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:03.865360 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:03.866521 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:03.866523 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:03.866523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:32:03.866528 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:03.866545 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:32:03.866550 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:03.866600 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:03.866675 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 20:32:03.866692 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 20:32:03.866953 9510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:03.864481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:45617 (local address 127.4.83.67:36741)
0619 20:32:03.864657 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:03.864691 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:03.865176 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:03.865430 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:03.865433 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:03.865441 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:03.866362 (+ 921us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:32:03.866363 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:03.866394 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:32:03.866425 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:03.866605 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:03.866610 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:03.866613 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:32:03.866681 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 20:32:03.866749 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":78,"threads_started":1}
I20260619 20:32:03.867447 9508 heartbeater.cc:347] Connected to a master server at 127.4.83.126:45617
I20260619 20:32:03.867528 9508 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:03.867674 9508 heartbeater.cc:511] Master 127.4.83.126:45617 requested a full tablet report, sending...
I20260619 20:32:03.868109 9058 ts_manager.cc:194] Registered new tserver with Master: 1f06008a99d14b069a6c97f8b2731d43 (127.4.83.67:33547)
I20260619 20:32:03.868953 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 9380
I20260619 20:32:03.869051 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance
I20260619 20:32:03.869222 4429 external_mini_cluster.cc:1428] Setting key 28e094c30fe31528cc2825d379ba7618
I20260619 20:32:03.869956 4429 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:32:03.872391 9098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:03.871171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54128 (local address 127.4.83.126:45617)
0619 20:32:03.871242 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:03.871244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:03.871322 (+ 78us) server_negotiation.cc:407] Connection header received
0619 20:32:03.871378 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:03.871379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:03.871383 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:03.871425 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:03.872070 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:03.872096 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:03.872097 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:32:03.872101 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:03.872116 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:32:03.872120 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:03.872175 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:03.872306 (+ 131us) server_negotiation.cc:299] Negotiation successful
0619 20:32:03.872316 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:32:03.874504 9058 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54128:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:32:03.876271 9058 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:32:03.880080 9516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:03.878519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:39541 (local address 127.0.0.1:51734)
0619 20:32:03.878801 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:03.878843 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:03.878915 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:03.879089 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:03.879093 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:03.879100 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:03.879653 (+ 553us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:32:03.879654 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:03.879671 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:32:03.879697 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:03.879906 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:03.879912 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:03.879915 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:32:03.879979 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 20:32:03.879993 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":74,"threads_started":1}
I20260619 20:32:03.880355 9519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:03.878624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51734 (local address 127.4.83.65:39541)
0619 20:32:03.878781 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:03.878787 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:03.878972 (+ 185us) server_negotiation.cc:407] Connection header received
0619 20:32:03.879001 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:03.879003 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:03.879014 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:03.879077 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:03.879763 (+ 686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:03.879771 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:03.879771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:32:03.879775 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:03.879810 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:32:03.879857 (+ 47us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:03.879913 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:03.880252 (+ 339us) server_negotiation.cc:299] Negotiation successful
0619 20:32:03.880269 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
I20260619 20:32:03.880460 9520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:03.878773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54942 (local address 127.4.83.66:39085)
0619 20:32:03.878946 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:03.878950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:03.879341 (+ 391us) server_negotiation.cc:407] Connection header received
0619 20:32:03.879384 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:03.879387 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:03.879397 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:03.879469 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:03.879967 (+ 498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:03.879974 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:03.879974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:32:03.879978 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:03.880020 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:32:03.880049 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:03.880118 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:03.880363 (+ 245us) server_negotiation.cc:299] Negotiation successful
0619 20:32:03.880380 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":81,"threads_started":1}
I20260619 20:32:03.880296 9518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:03.878521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:39085 (local address 127.0.0.1:54942)
0619 20:32:03.879092 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:03.879287 (+ 195us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:03.879333 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:03.879482 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:03.879482 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:03.879485 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:03.879879 (+ 394us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:32:03.879880 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:03.879887 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:32:03.879900 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:03.880129 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:03.880130 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:03.880131 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 20:32:03.880213 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 20:32:03.880226 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"mutex_wait_us":4,"thread_start_us":96,"threads_started":1}
I20260619 20:32:03.880772 9517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:03.878521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:33547 (local address 127.0.0.1:52028)
0619 20:32:03.878940 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:03.878950 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:03.879012 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:03.880035 (+ 1023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:03.880035 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:03.880038 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:03.880405 (+ 367us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:32:03.880405 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:03.880411 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:32:03.880423 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:03.880655 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:03.880656 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:03.880657 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 20:32:03.880699 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 20:32:03.880710 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"spinlock_wait_cycles":768,"thread_start_us":101,"threads_started":1}
I20260619 20:32:03.881139 9521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:03.878773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52028 (local address 127.4.83.67:33547)
0619 20:32:03.878952 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:03.878956 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:03.879004 (+ 48us) server_negotiation.cc:407] Connection header received
0619 20:32:03.879060 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:03.879063 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:03.879072 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:03.879138 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:03.880502 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:03.880511 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:03.880512 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:32:03.880517 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:03.880548 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:32:03.880572 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:03.880623 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:03.880913 (+ 290us) server_negotiation.cc:299] Negotiation successful
0619 20:32:03.880940 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":84,"threads_started":1}
I20260619 20:32:03.881384 9180 tablet_service.cc:1467] Processing CreateTablet for tablet 056c8465d8654740b1014a8d02e8c58d (DEFAULT_TABLE table=test-table [id=b2fad34b4c4b4967bf9923ea3ed05e32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:32:03.881486 9311 tablet_service.cc:1467] Processing CreateTablet for tablet 056c8465d8654740b1014a8d02e8c58d (DEFAULT_TABLE table=test-table [id=b2fad34b4c4b4967bf9923ea3ed05e32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:32:03.881795 9180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 056c8465d8654740b1014a8d02e8c58d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:03.881800 9311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 056c8465d8654740b1014a8d02e8c58d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:03.881723 9442 tablet_service.cc:1467] Processing CreateTablet for tablet 056c8465d8654740b1014a8d02e8c58d (DEFAULT_TABLE table=test-table [id=b2fad34b4c4b4967bf9923ea3ed05e32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:32:03.882000 9442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 056c8465d8654740b1014a8d02e8c58d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:03.884521 9522 tablet_bootstrap.cc:492] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485: Bootstrap starting.
I20260619 20:32:03.885105 9523 tablet_bootstrap.cc:492] T 056c8465d8654740b1014a8d02e8c58d P 1f06008a99d14b069a6c97f8b2731d43: Bootstrap starting.
I20260619 20:32:03.885157 9522 tablet_bootstrap.cc:654] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:03.885459 9522 log.cc:824] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:03.885813 9523 tablet_bootstrap.cc:654] T 056c8465d8654740b1014a8d02e8c58d P 1f06008a99d14b069a6c97f8b2731d43: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:03.886089 9523 log.cc:824] T 056c8465d8654740b1014a8d02e8c58d P 1f06008a99d14b069a6c97f8b2731d43: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:03.886422 9524 tablet_bootstrap.cc:492] T 056c8465d8654740b1014a8d02e8c58d P a7b9828576d64060bbbb696f322ff308: Bootstrap starting.
I20260619 20:32:03.886371 9522 tablet_bootstrap.cc:492] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485: No bootstrap required, opened a new log
I20260619 20:32:03.886633 9522 ts_tablet_manager.cc:1397] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:32:03.886811 9523 tablet_bootstrap.cc:492] T 056c8465d8654740b1014a8d02e8c58d P 1f06008a99d14b069a6c97f8b2731d43: No bootstrap required, opened a new log
I20260619 20:32:03.886875 9523 ts_tablet_manager.cc:1397] T 056c8465d8654740b1014a8d02e8c58d P 1f06008a99d14b069a6c97f8b2731d43: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:32:03.887123 9524 tablet_bootstrap.cc:654] T 056c8465d8654740b1014a8d02e8c58d P a7b9828576d64060bbbb696f322ff308: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:03.887445 9524 log.cc:824] T 056c8465d8654740b1014a8d02e8c58d P a7b9828576d64060bbbb696f322ff308: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:03.888266 9524 tablet_bootstrap.cc:492] T 056c8465d8654740b1014a8d02e8c58d P a7b9828576d64060bbbb696f322ff308: No bootstrap required, opened a new log
I20260619 20:32:03.888340 9524 ts_tablet_manager.cc:1397] T 056c8465d8654740b1014a8d02e8c58d P a7b9828576d64060bbbb696f322ff308: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:32:03.889089 9523 raft_consensus.cc:348] T 056c8465d8654740b1014a8d02e8c58d P 1f06008a99d14b069a6c97f8b2731d43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6c00d1d0ed748f2a4734f96a8fce485" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 39085 } } peers { permanent_uuid: "a7b9828576d64060bbbb696f322ff308" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39541 } } peers { permanent_uuid: "1f06008a99d14b069a6c97f8b2731d43" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 33547 } }
I20260619 20:32:03.889236 9523 raft_consensus.cc:374] T 056c8465d8654740b1014a8d02e8c58d P 1f06008a99d14b069a6c97f8b2731d43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:03.889267 9523 raft_consensus.cc:729] T 056c8465d8654740b1014a8d02e8c58d P 1f06008a99d14b069a6c97f8b2731d43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f06008a99d14b069a6c97f8b2731d43, State: Initialized, Role: FOLLOWER
I20260619 20:32:03.889401 9523 consensus_queue.cc:260] T 056c8465d8654740b1014a8d02e8c58d P 1f06008a99d14b069a6c97f8b2731d43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6c00d1d0ed748f2a4734f96a8fce485" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 39085 } } peers { permanent_uuid: "a7b9828576d64060bbbb696f322ff308" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39541 } } peers { permanent_uuid: "1f06008a99d14b069a6c97f8b2731d43" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 33547 } }
I20260619 20:32:03.889668 9523 ts_tablet_manager.cc:1428] T 056c8465d8654740b1014a8d02e8c58d P 1f06008a99d14b069a6c97f8b2731d43: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:32:03.889804 9508 heartbeater.cc:503] Master 127.4.83.126:45617 was elected leader, sending a full tablet report...
I20260619 20:32:03.889424 9522 raft_consensus.cc:348] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6c00d1d0ed748f2a4734f96a8fce485" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 39085 } } peers { permanent_uuid: "a7b9828576d64060bbbb696f322ff308" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39541 } } peers { permanent_uuid: "1f06008a99d14b069a6c97f8b2731d43" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 33547 } }
I20260619 20:32:03.890187 9522 raft_consensus.cc:374] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:03.890249 9522 raft_consensus.cc:729] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6c00d1d0ed748f2a4734f96a8fce485, State: Initialized, Role: FOLLOWER
I20260619 20:32:03.890381 9522 consensus_queue.cc:260] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6c00d1d0ed748f2a4734f96a8fce485" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 39085 } } peers { permanent_uuid: "a7b9828576d64060bbbb696f322ff308" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39541 } } peers { permanent_uuid: "1f06008a99d14b069a6c97f8b2731d43" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 33547 } }
I20260619 20:32:03.890650 9522 ts_tablet_manager.cc:1428] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260619 20:32:03.890726 9377 heartbeater.cc:503] Master 127.4.83.126:45617 was elected leader, sending a full tablet report...
I20260619 20:32:03.890877 9524 raft_consensus.cc:348] T 056c8465d8654740b1014a8d02e8c58d P a7b9828576d64060bbbb696f322ff308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6c00d1d0ed748f2a4734f96a8fce485" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 39085 } } peers { permanent_uuid: "a7b9828576d64060bbbb696f322ff308" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39541 } } peers { permanent_uuid: "1f06008a99d14b069a6c97f8b2731d43" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 33547 } }
I20260619 20:32:03.891017 9524 raft_consensus.cc:374] T 056c8465d8654740b1014a8d02e8c58d P a7b9828576d64060bbbb696f322ff308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:03.891093 9524 raft_consensus.cc:729] T 056c8465d8654740b1014a8d02e8c58d P a7b9828576d64060bbbb696f322ff308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7b9828576d64060bbbb696f322ff308, State: Initialized, Role: FOLLOWER
I20260619 20:32:03.891245 9524 consensus_queue.cc:260] T 056c8465d8654740b1014a8d02e8c58d P a7b9828576d64060bbbb696f322ff308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6c00d1d0ed748f2a4734f96a8fce485" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 39085 } } peers { permanent_uuid: "a7b9828576d64060bbbb696f322ff308" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39541 } } peers { permanent_uuid: "1f06008a99d14b069a6c97f8b2731d43" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 33547 } }
I20260619 20:32:03.891494 9524 ts_tablet_manager.cc:1428] T 056c8465d8654740b1014a8d02e8c58d P a7b9828576d64060bbbb696f322ff308: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:32:03.891523 9246 heartbeater.cc:503] Master 127.4.83.126:45617 was elected leader, sending a full tablet report...
W20260619 20:32:03.994393 9378 tablet.cc:2369] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:32:04.114391 9509 tablet.cc:2369] T 056c8465d8654740b1014a8d02e8c58d P 1f06008a99d14b069a6c97f8b2731d43: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:32:04.128273 9529 raft_consensus.cc:482] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:32:04.128404 9529 raft_consensus.cc:504] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6c00d1d0ed748f2a4734f96a8fce485" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 39085 } } peers { permanent_uuid: "a7b9828576d64060bbbb696f322ff308" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39541 } } peers { permanent_uuid: "1f06008a99d14b069a6c97f8b2731d43" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 33547 } }
I20260619 20:32:04.128772 9529 leader_election.cc:288] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7b9828576d64060bbbb696f322ff308 (127.4.83.65:39541), 1f06008a99d14b069a6c97f8b2731d43 (127.4.83.67:33547)
I20260619 20:32:04.129993 9379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:04.128972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:39541 (local address 127.4.83.66:54523)
0619 20:32:04.129037 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:04.129049 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:04.129093 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:04.129241 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:04.129242 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:04.129244 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:04.129673 (+ 429us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:32:04.129674 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:04.129682 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:32:04.129696 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:04.129863 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:04.129864 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:04.129864 (+ 0us) client_negotiation.cc:769] Sending connection context
0619 20:32:04.129908 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 20:32:04.129923 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 20:32:04.130004 9519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:04.129069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:54523 (local address 127.4.83.65:39541)
0619 20:32:04.129112 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:04.129114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:04.129123 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:32:04.129149 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:04.129150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:04.129155 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:04.129205 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:04.129776 (+ 571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:04.129778 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:04.129778 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:32:04.129782 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:04.129798 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:32:04.129801 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:04.129854 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:04.129931 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 20:32:04.129945 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:32:04.130225 9521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:04.129047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:58613 (local address 127.4.83.67:33547)
0619 20:32:04.129095 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:04.129098 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:04.129234 (+ 136us) server_negotiation.cc:407] Connection header received
0619 20:32:04.129282 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:04.129283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:04.129288 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:04.129335 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:04.129987 (+ 652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:04.129988 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:04.129988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:32:04.129993 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:04.130007 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:32:04.130011 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:04.130056 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:04.130133 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 20:32:04.130148 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 20:32:04.130395 9531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:04.128980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:33547 (local address 127.4.83.66:58613)
0619 20:32:04.129165 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:04.129174 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:04.129227 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:04.129337 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:04.129338 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:04.129351 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:04.129904 (+ 553us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:32:04.129904 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:04.129909 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:32:04.129919 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:04.130058 (+ 139us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:04.130059 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:04.130060 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 20:32:04.130312 (+ 252us) client_negotiation.cc:240] Negotiation successful
0619 20:32:04.130326 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":14,"thread_start_us":118,"threads_started":1}
I20260619 20:32:04.130398 9199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "056c8465d8654740b1014a8d02e8c58d" candidate_uuid: "a6c00d1d0ed748f2a4734f96a8fce485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7b9828576d64060bbbb696f322ff308" is_pre_election: true
I20260619 20:32:04.130545 9199 raft_consensus.cc:2454] T 056c8465d8654740b1014a8d02e8c58d P a7b9828576d64060bbbb696f322ff308 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6c00d1d0ed748f2a4734f96a8fce485 in term 0.
I20260619 20:32:04.130698 9462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "056c8465d8654740b1014a8d02e8c58d" candidate_uuid: "a6c00d1d0ed748f2a4734f96a8fce485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f06008a99d14b069a6c97f8b2731d43" is_pre_election: true
I20260619 20:32:04.130756 9267 leader_election.cc:302] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6c00d1d0ed748f2a4734f96a8fce485, a7b9828576d64060bbbb696f322ff308; no voters:
I20260619 20:32:04.130834 9462 raft_consensus.cc:2454] T 056c8465d8654740b1014a8d02e8c58d P 1f06008a99d14b069a6c97f8b2731d43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6c00d1d0ed748f2a4734f96a8fce485 in term 0.
I20260619 20:32:04.130865 9529 raft_consensus.cc:2781] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:32:04.130916 9529 raft_consensus.cc:482] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:32:04.130945 9529 raft_consensus.cc:3037] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:04.131681 9529 raft_consensus.cc:504] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6c00d1d0ed748f2a4734f96a8fce485" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 39085 } } peers { permanent_uuid: "a7b9828576d64060bbbb696f322ff308" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39541 } } peers { permanent_uuid: "1f06008a99d14b069a6c97f8b2731d43" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 33547 } }
I20260619 20:32:04.131798 9529 leader_election.cc:288] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485 [CANDIDATE]: Term 1 election: Requested vote from peers a7b9828576d64060bbbb696f322ff308 (127.4.83.65:39541), 1f06008a99d14b069a6c97f8b2731d43 (127.4.83.67:33547)
I20260619 20:32:04.131965 9199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "056c8465d8654740b1014a8d02e8c58d" candidate_uuid: "a6c00d1d0ed748f2a4734f96a8fce485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7b9828576d64060bbbb696f322ff308"
I20260619 20:32:04.131971 9462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "056c8465d8654740b1014a8d02e8c58d" candidate_uuid: "a6c00d1d0ed748f2a4734f96a8fce485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f06008a99d14b069a6c97f8b2731d43"
I20260619 20:32:04.132035 9199 raft_consensus.cc:3037] T 056c8465d8654740b1014a8d02e8c58d P a7b9828576d64060bbbb696f322ff308 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:04.132040 9462 raft_consensus.cc:3037] T 056c8465d8654740b1014a8d02e8c58d P 1f06008a99d14b069a6c97f8b2731d43 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:04.132697 9199 raft_consensus.cc:2454] T 056c8465d8654740b1014a8d02e8c58d P a7b9828576d64060bbbb696f322ff308 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6c00d1d0ed748f2a4734f96a8fce485 in term 1.
I20260619 20:32:04.132751 9462 raft_consensus.cc:2454] T 056c8465d8654740b1014a8d02e8c58d P 1f06008a99d14b069a6c97f8b2731d43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6c00d1d0ed748f2a4734f96a8fce485 in term 1.
I20260619 20:32:04.132880 9267 leader_election.cc:302] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6c00d1d0ed748f2a4734f96a8fce485, a7b9828576d64060bbbb696f322ff308; no voters:
I20260619 20:32:04.132992 9529 raft_consensus.cc:2781] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:32:04.133131 9529 raft_consensus.cc:686] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485 [term 1 LEADER]: Becoming Leader. State: Replica: a6c00d1d0ed748f2a4734f96a8fce485, State: Running, Role: LEADER
I20260619 20:32:04.133216 9529 consensus_queue.cc:237] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6c00d1d0ed748f2a4734f96a8fce485" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 39085 } } peers { permanent_uuid: "a7b9828576d64060bbbb696f322ff308" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39541 } } peers { permanent_uuid: "1f06008a99d14b069a6c97f8b2731d43" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 33547 } }
I20260619 20:32:04.134058 9058 catalog_manager.cc:5495] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485 reported cstate change: term changed from 0 to 1, leader changed from <none> to a6c00d1d0ed748f2a4734f96a8fce485 (127.4.83.66). New cstate: current_term: 1 leader_uuid: "a6c00d1d0ed748f2a4734f96a8fce485" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6c00d1d0ed748f2a4734f96a8fce485" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 39085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a7b9828576d64060bbbb696f322ff308" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f06008a99d14b069a6c97f8b2731d43" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 33547 } health_report { overall_health: UNKNOWN } } }
W20260619 20:32:04.135237 9247 tablet.cc:2369] T 056c8465d8654740b1014a8d02e8c58d P a7b9828576d64060bbbb696f322ff308: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:32:04.166834 9520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:04.165938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54958 (local address 127.4.83.66:39085)
0619 20:32:04.165990 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:04.165992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:04.165999 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:32:04.166052 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:04.166053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:04.166066 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:04.166112 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:04.166614 (+ 502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:04.166615 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:04.166615 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:32:04.166618 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:04.166632 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:32:04.166635 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:04.166678 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:04.166745 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 20:32:04.166757 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 20:32:04.168573 9199 raft_consensus.cc:1261] T 056c8465d8654740b1014a8d02e8c58d P a7b9828576d64060bbbb696f322ff308 [term 1 FOLLOWER]: Refusing update from remote peer a6c00d1d0ed748f2a4734f96a8fce485: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:32:04.168573 9462 raft_consensus.cc:1261] T 056c8465d8654740b1014a8d02e8c58d P 1f06008a99d14b069a6c97f8b2731d43 [term 1 FOLLOWER]: Refusing update from remote peer a6c00d1d0ed748f2a4734f96a8fce485: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:32:04.168910 9532 consensus_queue.cc:1035] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f06008a99d14b069a6c97f8b2731d43" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 33547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:32:04.169032 9529 consensus_queue.cc:1035] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7b9828576d64060bbbb696f322ff308" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:32:04.171507 9537 mvcc.cc:205] Tried to move back new op lower bound from 7298667004591763456 to 7298667004450127872. Current Snapshot: MvccSnapshot[applied={T|T < 7298667004591763456}]
I20260619 20:32:04.173569 9058 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54128:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:32:04.173686 9058 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54128:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:32:04.174474 9058 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 084e302719144ffa99ebfcda8b867dee: Sending DeleteTablet for 3 replicas of tablet 056c8465d8654740b1014a8d02e8c58d
I20260619 20:32:04.174844 9180 tablet_service.cc:1514] Processing DeleteTablet for tablet 056c8465d8654740b1014a8d02e8c58d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:32:04 UTC) from {username='slave'} at 127.0.0.1:51734
I20260619 20:32:04.174932 9311 tablet_service.cc:1514] Processing DeleteTablet for tablet 056c8465d8654740b1014a8d02e8c58d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:32:04 UTC) from {username='slave'} at 127.0.0.1:54942
I20260619 20:32:04.174953 9442 tablet_service.cc:1514] Processing DeleteTablet for tablet 056c8465d8654740b1014a8d02e8c58d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:32:04 UTC) from {username='slave'} at 127.0.0.1:52028
I20260619 20:32:04.175211 9546 tablet_replica.cc:330] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485: stopping tablet replica
I20260619 20:32:04.175318 9546 raft_consensus.cc:2229] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:32:04.175374 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 9118
I20260619 20:32:04.176118 9546 raft_consensus.cc:2258] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:32:04.177018 9546 ts_tablet_manager.cc:1905] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:32:04.177513 9547 tablet_replica.cc:330] T 056c8465d8654740b1014a8d02e8c58d P 1f06008a99d14b069a6c97f8b2731d43: stopping tablet replica
I20260619 20:32:04.177634 9547 raft_consensus.cc:2229] T 056c8465d8654740b1014a8d02e8c58d P 1f06008a99d14b069a6c97f8b2731d43 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:32:04.177747 9547 raft_consensus.cc:2258] T 056c8465d8654740b1014a8d02e8c58d P 1f06008a99d14b069a6c97f8b2731d43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:32:04.178262 9547 ts_tablet_manager.cc:1905] T 056c8465d8654740b1014a8d02e8c58d P 1f06008a99d14b069a6c97f8b2731d43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:32:04.178843 9546 ts_tablet_manager.cc:1918] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:32:04.178966 9546 log.cc:1196] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-1/wal/wals/056c8465d8654740b1014a8d02e8c58d
I20260619 20:32:04.179328 9546 ts_tablet_manager.cc:1939] T 056c8465d8654740b1014a8d02e8c58d P a6c00d1d0ed748f2a4734f96a8fce485: Deleting consensus metadata
I20260619 20:32:04.179819 9042 catalog_manager.cc:4841] TS a6c00d1d0ed748f2a4734f96a8fce485 (127.4.83.66:39085): tablet 056c8465d8654740b1014a8d02e8c58d (table test-table [id=b2fad34b4c4b4967bf9923ea3ed05e32]) successfully deleted
I20260619 20:32:04.179924 9547 ts_tablet_manager.cc:1918] T 056c8465d8654740b1014a8d02e8c58d P 1f06008a99d14b069a6c97f8b2731d43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:32:04.179986 9547 log.cc:1196] T 056c8465d8654740b1014a8d02e8c58d P 1f06008a99d14b069a6c97f8b2731d43: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901102012824-4429-0/minicluster-data/ts-2/wal/wals/056c8465d8654740b1014a8d02e8c58d
I20260619 20:32:04.180277 9547 ts_tablet_manager.cc:1939] T 056c8465d8654740b1014a8d02e8c58d P 1f06008a99d14b069a6c97f8b2731d43: Deleting consensus metadata
I20260619 20:32:04.180895 9044 catalog_manager.cc:4841] TS 1f06008a99d14b069a6c97f8b2731d43 (127.4.83.67:33547): tablet 056c8465d8654740b1014a8d02e8c58d (table test-table [id=b2fad34b4c4b4967bf9923ea3ed05e32]) successfully deleted
W20260619 20:32:04.181764 9045 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.83.65:39541 (error 108)
I20260619 20:32:04.181958 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 9249
I20260619 20:32:04.182571 9517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:04.182246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:39541 (local address 127.0.0.1:51744)
0619 20:32:04.182478 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:04.182535 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.65:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206}
W20260619 20:32:04.182709 9045 catalog_manager.cc:4584] TS a7b9828576d64060bbbb696f322ff308 (127.4.83.65:39541): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 056c8465d8654740b1014a8d02e8c58d: Network error: Client connection negotiation failed: client connection to 127.4.83.65:39541: connect: Connection refused (error 111)
I20260619 20:32:04.186638 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 9380
I20260619 20:32:04.191357 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 9027
2026-06-19T20:32:04Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (846 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260619 20:32:04.210001 4429 mini_oidc.cc:152] Starting JWKS server
I20260619 20:32:04.211175 4429 webserver.cc:460] Webserver started at https://127.0.0.1:34351/ using document root <none> and password file <none>
I20260619 20:32:04.211268 4429 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 20:32:04.211434 4429 webserver.cc:460] Webserver started at http://127.0.0.1:36093/ using document root <none> and password file <none>
2026-06-19T20:32:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:32:04Z Disabled control of system clock
I20260619 20:32:04.219177 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:37875
--webserver_interface=127.4.83.126
--webserver_port=0
--builtin_ntp_servers=127.4.83.84:46695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:37875
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34351/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901102012824-4429-0/testchainca.cert with env {}
W20260619 20:32:04.301476 9554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:04.301714 9554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:04.301738 9554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:04.302912 9554 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:32:04.302949 9554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:04.302960 9554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:04.302973 9554 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:32:04.302983 9554 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 20:32:04.302995 9554 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 20:32:04.303004 9554 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34351/jwks.json
I20260619 20:32:04.304616 9554 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:46695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:37875
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:37875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34351/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901102012824-4429-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:04.304857 9554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:04.305029 9554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:04.310123 9559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:04.310149 9562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:04.310127 9560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:04.310324 9554 server_base.cc:1034] running on GCE node
I20260619 20:32:04.310657 9554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:04.310988 9554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:04.312177 9554 hybrid_clock.cc:648] HybridClock initialized: now 1781901124312164 us; error 32 us; skew 500 ppm
I20260619 20:32:04.313587 9554 webserver.cc:460] Webserver started at http://127.4.83.126:43969/ using document root <none> and password file <none>
I20260619 20:32:04.313874 9554 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:04.313941 9554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:04.314040 9554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:04.315598 9554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "ed074b24597546318f783629e1c36d14"
format_stamp: "Formatted at 2026-06-19 20:32:04 on dist-test-slave-4m58"
server_key: "e169f56f9ea51e27db671a2fbc5fef00"
server_key_iv: "fc4ccccc094bfb177b5c9f1971f62de6"
server_key_version: "encryptionkey@0"
I20260619 20:32:04.316051 9554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "ed074b24597546318f783629e1c36d14"
format_stamp: "Formatted at 2026-06-19 20:32:04 on dist-test-slave-4m58"
server_key: "e169f56f9ea51e27db671a2fbc5fef00"
server_key_iv: "fc4ccccc094bfb177b5c9f1971f62de6"
server_key_version: "encryptionkey@0"
I20260619 20:32:04.317390 9554 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:32:04.318225 9568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:04.318532 9554 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:32:04.318643 9554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "ed074b24597546318f783629e1c36d14"
format_stamp: "Formatted at 2026-06-19 20:32:04 on dist-test-slave-4m58"
server_key: "e169f56f9ea51e27db671a2fbc5fef00"
server_key_iv: "fc4ccccc094bfb177b5c9f1971f62de6"
server_key_version: "encryptionkey@0"
I20260619 20:32:04.318724 9554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:04.348855 9554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:04.349203 9554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:04.349340 9554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:04.353042 9554 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:37875
I20260619 20:32:04.353138 9620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:37875 every 8 connection(s)
I20260619 20:32:04.353422 9554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:32:04.353912 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 9554
I20260619 20:32:04.354007 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:32:04.354179 4429 external_mini_cluster.cc:1428] Setting key cb43df45b48f340df14d30059675c52a
I20260619 20:32:04.354447 9621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:04.357957 9621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed074b24597546318f783629e1c36d14: Bootstrap starting.
I20260619 20:32:04.358750 9621 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed074b24597546318f783629e1c36d14: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:04.359081 9621 log.cc:824] T 00000000000000000000000000000000 P ed074b24597546318f783629e1c36d14: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:04.359916 9621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed074b24597546318f783629e1c36d14: No bootstrap required, opened a new log
I20260619 20:32:04.360530 9624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:04.354708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56720 (local address 127.4.83.126:37875)
0619 20:32:04.354939 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:04.354949 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:04.354982 (+ 33us) server_negotiation.cc:407] Connection header received
0619 20:32:04.355635 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:04.355646 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:04.355734 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:04.355936 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:04.356113 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:04.357430 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:04.358358 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:04.358982 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:04.359114 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:04.359815 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:04.359819 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:04.359819 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:32:04.360054 (+ 235us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:04.360095 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:32:04.360111 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:04.360170 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:04.360244 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 20:32:04.360324 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":102,"threads_started":1}
I20260619 20:32:04.361465 4429 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:32:04.362223 9621 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed074b24597546318f783629e1c36d14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed074b24597546318f783629e1c36d14" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37875 } }
I20260619 20:32:04.362385 9621 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed074b24597546318f783629e1c36d14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:04.362425 9621 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed074b24597546318f783629e1c36d14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed074b24597546318f783629e1c36d14, State: Initialized, Role: FOLLOWER
I20260619 20:32:04.362536 9621 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed074b24597546318f783629e1c36d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed074b24597546318f783629e1c36d14" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37875 } }
I20260619 20:32:04.362614 9621 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed074b24597546318f783629e1c36d14 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:32:04.362638 9621 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed074b24597546318f783629e1c36d14 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:32:04.362670 9621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed074b24597546318f783629e1c36d14 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:04.363199 9621 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed074b24597546318f783629e1c36d14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed074b24597546318f783629e1c36d14" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37875 } }
I20260619 20:32:04.363281 9621 leader_election.cc:302] T 00000000000000000000000000000000 P ed074b24597546318f783629e1c36d14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed074b24597546318f783629e1c36d14; no voters:
I20260619 20:32:04.363401 9621 leader_election.cc:288] T 00000000000000000000000000000000 P ed074b24597546318f783629e1c36d14 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:32:04.363444 9626 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed074b24597546318f783629e1c36d14 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:32:04.363583 9626 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed074b24597546318f783629e1c36d14 [term 1 LEADER]: Becoming Leader. State: Replica: ed074b24597546318f783629e1c36d14, State: Running, Role: LEADER
I20260619 20:32:04.363626 9621 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed074b24597546318f783629e1c36d14 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:32:04.363664 9626 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed074b24597546318f783629e1c36d14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed074b24597546318f783629e1c36d14" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37875 } }
I20260619 20:32:04.364065 9627 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed074b24597546318f783629e1c36d14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed074b24597546318f783629e1c36d14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed074b24597546318f783629e1c36d14" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37875 } } }
I20260619 20:32:04.364079 9628 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed074b24597546318f783629e1c36d14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed074b24597546318f783629e1c36d14. Latest consensus state: current_term: 1 leader_uuid: "ed074b24597546318f783629e1c36d14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed074b24597546318f783629e1c36d14" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37875 } } }
I20260619 20:32:04.364157 9627 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed074b24597546318f783629e1c36d14 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:04.364164 9628 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed074b24597546318f783629e1c36d14 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:04.364341 9633 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:32:04.364822 9633 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:32:04.366256 9633 catalog_manager.cc:1347] Generated new cluster ID: d88466d056254a079e7b40b428cfc1e1
I20260619 20:32:04.366298 9633 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:32:04.377528 9633 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:32:04.378121 9633 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:32:04.386564 9633 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed074b24597546318f783629e1c36d14: Generated new TSK 0
I20260619 20:32:04.386935 9633 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 20:32:04.423295 9651 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
I20260619 20:32:04.423516 9624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:04.400516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56726 (local address 127.4.83.126:37875)
0619 20:32:04.400562 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:04.400564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:04.400692 (+ 128us) server_negotiation.cc:407] Connection header received
0619 20:32:04.400786 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:04.400787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:04.400800 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:04.400862 (+ 62us) server_negotiation.cc:226] Negotiated authn=JWT
0619 20:32:04.401057 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:04.401562 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:04.402711 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:04.402900 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:04.402955 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:04.403175 (+ 220us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 20:32:04.423297 (+ 20122us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34351/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 20:32:04.423441 (+ 144us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56726: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34351/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":25}
E20260619 20:32:04.423564 9651 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260619 20:32:04.423856 9624 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56726: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34351/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260619 20:32:04.423916 9650 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:04.400431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:37875 (local address 127.0.0.1:56726)
0619 20:32:04.400596 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:04.400609 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:04.400684 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:04.400879 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:04.400880 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:04.400882 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 20:32:04.400956 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:04.400959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:04.401660 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:04.401662 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:04.402621 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:04.402624 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:04.402971 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:04.402972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:04.403038 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:04.403066 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 20:32:04.423433 (+ 20367us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34351/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 20:32:04.423564 (+ 131us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.83.126:37875: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34351/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":71,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.4.83.126:37875: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34351/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260619 20:32:04.424396 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 9554
2026-06-19T20:32:04Z chronyd exiting
I20260619 20:32:04.460475 4429 test_util.cc:175] -----------------------------------------------
I20260619 20:32:04.460561 4429 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901102012824-4429-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (251 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260619 20:32:04.461382 4429 mini_oidc.cc:152] Starting JWKS server
I20260619 20:32:04.462055 4429 webserver.cc:460] Webserver started at https://127.0.0.1:38135/ using document root <none> and password file <none>
I20260619 20:32:04.462096 4429 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 20:32:04.462224 4429 webserver.cc:460] Webserver started at http://127.0.0.1:41999/ using document root <none> and password file <none>
2026-06-19T20:32:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:32:04Z Disabled control of system clock
I20260619 20:32:04.469753 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:46285
--webserver_interface=127.4.83.126
--webserver_port=0
--builtin_ntp_servers=127.4.83.84:38375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:46285
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38135/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901102012824-4429-0/testchainca.cert with env {}
W20260619 20:32:04.550228 9657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:04.550453 9657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:04.550483 9657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:04.551646 9657 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:32:04.551692 9657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:04.551707 9657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:04.551724 9657 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:32:04.551740 9657 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 20:32:04.551764 9657 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 20:32:04.551779 9657 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38135/jwks.json
I20260619 20:32:04.553385 9657 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:38375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:46285
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:46285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38135/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901102012824-4429-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:04.553648 9657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:04.553854 9657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:04.558807 9663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:04.558797 9662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:04.558822 9657 server_base.cc:1034] running on GCE node
W20260619 20:32:04.558797 9665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:04.559273 9657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:04.559559 9657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:04.560708 9657 hybrid_clock.cc:648] HybridClock initialized: now 1781901124560692 us; error 36 us; skew 500 ppm
I20260619 20:32:04.561935 9657 webserver.cc:460] Webserver started at http://127.4.83.126:37617/ using document root <none> and password file <none>
I20260619 20:32:04.562146 9657 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:04.562194 9657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:04.562316 9657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:04.563388 9657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "eac5e7d4e716480ca83d91a39a12b705"
format_stamp: "Formatted at 2026-06-19 20:32:04 on dist-test-slave-4m58"
server_key: "a502184042d3b1e9ebb47ba3a7d78b3e"
server_key_iv: "c196efa610889e7ffffa1c312bbe160d"
server_key_version: "encryptionkey@0"
I20260619 20:32:04.563717 9657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "eac5e7d4e716480ca83d91a39a12b705"
format_stamp: "Formatted at 2026-06-19 20:32:04 on dist-test-slave-4m58"
server_key: "a502184042d3b1e9ebb47ba3a7d78b3e"
server_key_iv: "c196efa610889e7ffffa1c312bbe160d"
server_key_version: "encryptionkey@0"
I20260619 20:32:04.565021 9657 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:32:04.565901 9671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:04.566077 9657 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:32:04.566164 9657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "eac5e7d4e716480ca83d91a39a12b705"
format_stamp: "Formatted at 2026-06-19 20:32:04 on dist-test-slave-4m58"
server_key: "a502184042d3b1e9ebb47ba3a7d78b3e"
server_key_iv: "c196efa610889e7ffffa1c312bbe160d"
server_key_version: "encryptionkey@0"
I20260619 20:32:04.566251 9657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:04.584522 9657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:04.584857 9657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:04.584991 9657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:04.588920 9657 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:46285
I20260619 20:32:04.588953 9723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:46285 every 8 connection(s)
I20260619 20:32:04.589327 9657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:32:04.590409 9724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:04.592814 9724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eac5e7d4e716480ca83d91a39a12b705: Bootstrap starting.
I20260619 20:32:04.593400 9724 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eac5e7d4e716480ca83d91a39a12b705: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:04.593652 9724 log.cc:824] T 00000000000000000000000000000000 P eac5e7d4e716480ca83d91a39a12b705: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:04.594403 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 9657
I20260619 20:32:04.594460 9724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eac5e7d4e716480ca83d91a39a12b705: No bootstrap required, opened a new log
I20260619 20:32:04.594517 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:32:04.594767 4429 external_mini_cluster.cc:1428] Setting key 8f28326a68f99bc3c19e51898dfda114
I20260619 20:32:04.596415 9724 raft_consensus.cc:348] T 00000000000000000000000000000000 P eac5e7d4e716480ca83d91a39a12b705 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eac5e7d4e716480ca83d91a39a12b705" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 46285 } }
I20260619 20:32:04.596554 9724 raft_consensus.cc:374] T 00000000000000000000000000000000 P eac5e7d4e716480ca83d91a39a12b705 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:04.596575 9724 raft_consensus.cc:729] T 00000000000000000000000000000000 P eac5e7d4e716480ca83d91a39a12b705 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac5e7d4e716480ca83d91a39a12b705, State: Initialized, Role: FOLLOWER
I20260619 20:32:04.596715 9724 consensus_queue.cc:260] T 00000000000000000000000000000000 P eac5e7d4e716480ca83d91a39a12b705 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eac5e7d4e716480ca83d91a39a12b705" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 46285 } }
I20260619 20:32:04.596800 9724 raft_consensus.cc:388] T 00000000000000000000000000000000 P eac5e7d4e716480ca83d91a39a12b705 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:32:04.596832 9724 raft_consensus.cc:482] T 00000000000000000000000000000000 P eac5e7d4e716480ca83d91a39a12b705 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:32:04.596880 9724 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eac5e7d4e716480ca83d91a39a12b705 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:04.597461 9724 raft_consensus.cc:504] T 00000000000000000000000000000000 P eac5e7d4e716480ca83d91a39a12b705 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eac5e7d4e716480ca83d91a39a12b705" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 46285 } }
I20260619 20:32:04.597548 9724 leader_election.cc:302] T 00000000000000000000000000000000 P eac5e7d4e716480ca83d91a39a12b705 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eac5e7d4e716480ca83d91a39a12b705; no voters:
I20260619 20:32:04.597695 9724 leader_election.cc:288] T 00000000000000000000000000000000 P eac5e7d4e716480ca83d91a39a12b705 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:32:04.597728 9729 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eac5e7d4e716480ca83d91a39a12b705 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:32:04.597872 9729 raft_consensus.cc:686] T 00000000000000000000000000000000 P eac5e7d4e716480ca83d91a39a12b705 [term 1 LEADER]: Becoming Leader. State: Replica: eac5e7d4e716480ca83d91a39a12b705, State: Running, Role: LEADER
I20260619 20:32:04.597962 9724 sys_catalog.cc:564] T 00000000000000000000000000000000 P eac5e7d4e716480ca83d91a39a12b705 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:32:04.598001 9729 consensus_queue.cc:237] T 00000000000000000000000000000000 P eac5e7d4e716480ca83d91a39a12b705 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eac5e7d4e716480ca83d91a39a12b705" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 46285 } }
I20260619 20:32:04.598351 9729 sys_catalog.cc:455] T 00000000000000000000000000000000 P eac5e7d4e716480ca83d91a39a12b705 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eac5e7d4e716480ca83d91a39a12b705. Latest consensus state: current_term: 1 leader_uuid: "eac5e7d4e716480ca83d91a39a12b705" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eac5e7d4e716480ca83d91a39a12b705" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 46285 } } }
I20260619 20:32:04.598409 9730 sys_catalog.cc:455] T 00000000000000000000000000000000 P eac5e7d4e716480ca83d91a39a12b705 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eac5e7d4e716480ca83d91a39a12b705" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eac5e7d4e716480ca83d91a39a12b705" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 46285 } } }
I20260619 20:32:04.598484 9729 sys_catalog.cc:458] T 00000000000000000000000000000000 P eac5e7d4e716480ca83d91a39a12b705 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:04.598492 9730 sys_catalog.cc:458] T 00000000000000000000000000000000 P eac5e7d4e716480ca83d91a39a12b705 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:04.598734 9737 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:32:04.599300 9737 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:32:04.601795 9728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:04.595315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41026 (local address 127.4.83.126:46285)
0619 20:32:04.595500 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:04.595508 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:04.595531 (+ 23us) server_negotiation.cc:407] Connection header received
0619 20:32:04.596411 (+ 880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:04.596421 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:04.596487 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:04.596642 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:04.596943 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:04.598676 (+ 1733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:04.599870 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:04.600376 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:04.600519 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:04.601179 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:04.601182 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:04.601183 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:32:04.601361 (+ 178us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:04.601393 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:32:04.601403 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:04.601456 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:04.601539 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 20:32:04.601618 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":69,"threads_started":1}
I20260619 20:32:04.602687 4429 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:32:04.604313 9737 catalog_manager.cc:1347] Generated new cluster ID: 933e92880a174f509862ddaa162a1299
I20260619 20:32:04.604373 9737 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:32:04.627398 9737 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:32:04.628221 9737 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:32:04.645107 9737 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eac5e7d4e716480ca83d91a39a12b705: Generated new TSK 0
I20260619 20:32:04.645351 9737 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 20:32:04.677623 9754 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260619 20:32:04.677806 9754 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260619 20:32:04.678251 9753 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:04.653887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:46285 (local address 127.0.0.1:41042)
0619 20:32:04.654027 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:04.654038 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:04.654125 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:04.654315 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:04.654316 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:04.654318 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 20:32:04.654380 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:04.654383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:04.655397 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:04.655398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:04.656406 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:04.656409 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:04.656942 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:04.656943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:04.657009 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:04.657015 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 20:32:04.678004 (+ 20989us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38135/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 20:32:04.678141 (+ 137us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.83.126:46285: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38135/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
I20260619 20:32:04.678632 9728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:04.653952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41042 (local address 127.4.83.126:46285)
0619 20:32:04.654003 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:04.654005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:04.654114 (+ 109us) server_negotiation.cc:407] Connection header received
0619 20:32:04.654206 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:04.654208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:04.654221 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:04.654302 (+ 81us) server_negotiation.cc:226] Negotiated authn=JWT
0619 20:32:04.654442 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:04.655288 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:04.656523 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:04.656859 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:04.656937 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:04.657140 (+ 203us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 20:32:04.677840 (+ 20700us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38135/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 20:32:04.678552 (+ 712us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41042: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38135/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 20:32:04.678718 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 9657
2026-06-19T20:32:04Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (254 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260619 20:32:04.715811 4429 mini_oidc.cc:152] Starting JWKS server
I20260619 20:32:04.716553 4429 webserver.cc:460] Webserver started at https://127.0.0.1:39651/ using document root <none> and password file <none>
I20260619 20:32:04.716600 4429 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 20:32:04.716709 4429 webserver.cc:460] Webserver started at http://127.0.0.1:44457/ using document root <none> and password file <none>
2026-06-19T20:32:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:32:04Z Disabled control of system clock
I20260619 20:32:04.724479 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:40101
--webserver_interface=127.4.83.126
--webserver_port=0
--builtin_ntp_servers=127.4.83.84:35147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:40101
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39651/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 20:32:04.803992 9760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:04.804221 9760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:04.804243 9760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:04.805471 9760 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:32:04.805511 9760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:04.805522 9760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:04.805533 9760 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:32:04.805544 9760 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 20:32:04.805554 9760 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 20:32:04.805564 9760 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39651/jwks.json
I20260619 20:32:04.806934 9760 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:35147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:40101
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:40101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39651/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:04.807160 9760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:04.807411 9760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:04.812147 9768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:04.812317 9765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:04.812181 9766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:04.812455 9760 server_base.cc:1034] running on GCE node
I20260619 20:32:04.812671 9760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:04.812960 9760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:04.814133 9760 hybrid_clock.cc:648] HybridClock initialized: now 1781901124814116 us; error 32 us; skew 500 ppm
I20260619 20:32:04.815491 9760 webserver.cc:460] Webserver started at http://127.4.83.126:37395/ using document root <none> and password file <none>
I20260619 20:32:04.815737 9760 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:04.815784 9760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:04.815933 9760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:04.817039 9760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "de3a798d0c5b4a25bb7053457da2f90e"
format_stamp: "Formatted at 2026-06-19 20:32:04 on dist-test-slave-4m58"
server_key: "92ddeb0525f2ff67f1519117804c0567"
server_key_iv: "9db7c68240a2c7a752e344a1ef8f7420"
server_key_version: "encryptionkey@0"
I20260619 20:32:04.817363 9760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "de3a798d0c5b4a25bb7053457da2f90e"
format_stamp: "Formatted at 2026-06-19 20:32:04 on dist-test-slave-4m58"
server_key: "92ddeb0525f2ff67f1519117804c0567"
server_key_iv: "9db7c68240a2c7a752e344a1ef8f7420"
server_key_version: "encryptionkey@0"
I20260619 20:32:04.818607 9760 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:32:04.819376 9774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:04.819749 9760 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:04.819880 9760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "de3a798d0c5b4a25bb7053457da2f90e"
format_stamp: "Formatted at 2026-06-19 20:32:04 on dist-test-slave-4m58"
server_key: "92ddeb0525f2ff67f1519117804c0567"
server_key_iv: "9db7c68240a2c7a752e344a1ef8f7420"
server_key_version: "encryptionkey@0"
I20260619 20:32:04.819967 9760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:04.862294 9760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:04.862646 9760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:04.862782 9760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:04.866609 9760 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:40101
I20260619 20:32:04.866649 9826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:40101 every 8 connection(s)
I20260619 20:32:04.866978 9760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:32:04.867671 9827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:04.869390 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 9760
I20260619 20:32:04.869499 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:32:04.869673 4429 external_mini_cluster.cc:1428] Setting key b8f7c12f0fd8d54ddb7bbb3daa662f4d
I20260619 20:32:04.870715 9827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de3a798d0c5b4a25bb7053457da2f90e: Bootstrap starting.
I20260619 20:32:04.871361 9827 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de3a798d0c5b4a25bb7053457da2f90e: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:04.871627 9827 log.cc:824] T 00000000000000000000000000000000 P de3a798d0c5b4a25bb7053457da2f90e: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:04.872375 9827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de3a798d0c5b4a25bb7053457da2f90e: No bootstrap required, opened a new log
I20260619 20:32:04.874150 9827 raft_consensus.cc:348] T 00000000000000000000000000000000 P de3a798d0c5b4a25bb7053457da2f90e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3a798d0c5b4a25bb7053457da2f90e" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 40101 } }
I20260619 20:32:04.874291 9827 raft_consensus.cc:374] T 00000000000000000000000000000000 P de3a798d0c5b4a25bb7053457da2f90e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:04.874321 9827 raft_consensus.cc:729] T 00000000000000000000000000000000 P de3a798d0c5b4a25bb7053457da2f90e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de3a798d0c5b4a25bb7053457da2f90e, State: Initialized, Role: FOLLOWER
I20260619 20:32:04.874465 9827 consensus_queue.cc:260] T 00000000000000000000000000000000 P de3a798d0c5b4a25bb7053457da2f90e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3a798d0c5b4a25bb7053457da2f90e" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 40101 } }
I20260619 20:32:04.874533 9827 raft_consensus.cc:388] T 00000000000000000000000000000000 P de3a798d0c5b4a25bb7053457da2f90e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:32:04.874580 9827 raft_consensus.cc:482] T 00000000000000000000000000000000 P de3a798d0c5b4a25bb7053457da2f90e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:32:04.874635 9827 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de3a798d0c5b4a25bb7053457da2f90e [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:04.875049 9830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:04.870206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56200 (local address 127.4.83.126:40101)
0619 20:32:04.870393 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:04.870401 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:04.870428 (+ 27us) server_negotiation.cc:407] Connection header received
0619 20:32:04.870876 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:04.870883 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:04.870946 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:04.871070 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:04.871355 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:04.872455 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:04.873227 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:04.873643 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:04.873758 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:04.874387 (+ 629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:04.874391 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:04.874391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:32:04.874597 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:04.874639 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:32:04.874654 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:04.874713 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:04.874795 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 20:32:04.874874 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":74,"threads_started":1}
I20260619 20:32:04.875339 9827 raft_consensus.cc:504] T 00000000000000000000000000000000 P de3a798d0c5b4a25bb7053457da2f90e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3a798d0c5b4a25bb7053457da2f90e" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 40101 } }
I20260619 20:32:04.875439 9827 leader_election.cc:302] T 00000000000000000000000000000000 P de3a798d0c5b4a25bb7053457da2f90e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de3a798d0c5b4a25bb7053457da2f90e; no voters:
I20260619 20:32:04.875607 9827 leader_election.cc:288] T 00000000000000000000000000000000 P de3a798d0c5b4a25bb7053457da2f90e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:32:04.875638 9832 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de3a798d0c5b4a25bb7053457da2f90e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:32:04.875741 9832 raft_consensus.cc:686] T 00000000000000000000000000000000 P de3a798d0c5b4a25bb7053457da2f90e [term 1 LEADER]: Becoming Leader. State: Replica: de3a798d0c5b4a25bb7053457da2f90e, State: Running, Role: LEADER
I20260619 20:32:04.875888 4429 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:32:04.875890 9832 consensus_queue.cc:237] T 00000000000000000000000000000000 P de3a798d0c5b4a25bb7053457da2f90e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3a798d0c5b4a25bb7053457da2f90e" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 40101 } }
I20260619 20:32:04.875901 9827 sys_catalog.cc:564] T 00000000000000000000000000000000 P de3a798d0c5b4a25bb7053457da2f90e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:32:04.876277 9834 sys_catalog.cc:455] T 00000000000000000000000000000000 P de3a798d0c5b4a25bb7053457da2f90e [sys.catalog]: SysCatalogTable state changed. Reason: New leader de3a798d0c5b4a25bb7053457da2f90e. Latest consensus state: current_term: 1 leader_uuid: "de3a798d0c5b4a25bb7053457da2f90e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3a798d0c5b4a25bb7053457da2f90e" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 40101 } } }
I20260619 20:32:04.876261 9833 sys_catalog.cc:455] T 00000000000000000000000000000000 P de3a798d0c5b4a25bb7053457da2f90e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de3a798d0c5b4a25bb7053457da2f90e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de3a798d0c5b4a25bb7053457da2f90e" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 40101 } } }
I20260619 20:32:04.876400 9833 sys_catalog.cc:458] T 00000000000000000000000000000000 P de3a798d0c5b4a25bb7053457da2f90e [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:04.876391 9834 sys_catalog.cc:458] T 00000000000000000000000000000000 P de3a798d0c5b4a25bb7053457da2f90e [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:04.876613 9839 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:32:04.877084 9839 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:32:04.878432 9839 catalog_manager.cc:1347] Generated new cluster ID: 28b22921b6914d77b34e762313760afb
I20260619 20:32:04.878472 9839 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:32:04.910753 9839 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:32:04.911334 9839 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:32:04.914601 9839 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de3a798d0c5b4a25bb7053457da2f90e: Generated new TSK 0
I20260619 20:32:04.914820 9839 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:32:04.926882 9830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:04.923031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56214 (local address 127.4.83.126:40101)
0619 20:32:04.923095 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:04.923097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:04.923192 (+ 95us) server_negotiation.cc:407] Connection header received
0619 20:32:04.923281 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:04.923282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:04.923293 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:04.923357 (+ 64us) server_negotiation.cc:226] Negotiated authn=JWT
0619 20:32:04.923503 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:04.924164 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:04.925331 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:04.925528 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:04.925581 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:04.925802 (+ 221us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 20:32:04.926666 (+ 864us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39651/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 20:32:04.926802 (+ 136us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56214: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39651/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":36}
W20260619 20:32:04.926960 9830 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56214: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39651/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260619 20:32:04.926997 9856 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:04.922955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:40101 (local address 127.0.0.1:56214)
0619 20:32:04.923116 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:04.923128 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:04.923204 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:04.923370 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:04.923371 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:04.923373 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 20:32:04.923438 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:04.923440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:04.924269 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:04.924270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:04.925247 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:04.925250 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:04.925602 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:04.925603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:04.925664 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:04.925671 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 20:32:04.926772 (+ 1101us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39651/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 20:32:04.926891 (+ 119us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.83.126:40101: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39651/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":96,"threads_started":1}
I20260619 20:32:04.927249 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 9760
E20260619 20:32:04.942045 9857 webserver.cc:533] Webserver: sslize failed:
E20260619 20:32:04.942137 9857 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-06-19T20:32:04Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (248 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-19T20:32:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:32:04Z Disabled control of system clock
I20260619 20:32:04.972020 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:34433
--webserver_interface=127.4.83.126
--webserver_port=0
--builtin_ntp_servers=127.4.83.84:44245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:34433
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901102012824-4429-0/testchainca.cert with env {}
W20260619 20:32:05.056628 9861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:05.056851 9861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:05.056875 9861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:05.058044 9861 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:32:05.058084 9861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:05.058095 9861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:05.058106 9861 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:32:05.058118 9861 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 20:32:05.058139 9861 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 20:32:05.058149 9861 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260619 20:32:05.059661 9861 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:44245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:34433
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:34433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901102012824-4429-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:05.059907 9861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:05.060101 9861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:05.065271 9866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:05.065321 9869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:05.065282 9861 server_base.cc:1034] running on GCE node
W20260619 20:32:05.065282 9867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:05.065768 9861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:05.066063 9861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:05.067241 9861 hybrid_clock.cc:648] HybridClock initialized: now 1781901125067219 us; error 44 us; skew 500 ppm
I20260619 20:32:05.068802 9861 webserver.cc:460] Webserver started at http://127.4.83.126:40955/ using document root <none> and password file <none>
I20260619 20:32:05.069155 9861 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:05.069217 9861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:05.069343 9861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:05.070417 9861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "9f13bfb2d81448dba5f643920f634592"
format_stamp: "Formatted at 2026-06-19 20:32:05 on dist-test-slave-4m58"
server_key: "d816afddbeba5d9faded5a8597ffed3e"
server_key_iv: "fcd487d72e1931473c4e146fff3d6d19"
server_key_version: "encryptionkey@0"
I20260619 20:32:05.070760 9861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "9f13bfb2d81448dba5f643920f634592"
format_stamp: "Formatted at 2026-06-19 20:32:05 on dist-test-slave-4m58"
server_key: "d816afddbeba5d9faded5a8597ffed3e"
server_key_iv: "fcd487d72e1931473c4e146fff3d6d19"
server_key_version: "encryptionkey@0"
I20260619 20:32:05.072124 9861 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:32:05.072942 9875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:05.073158 9861 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:05.073256 9861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "9f13bfb2d81448dba5f643920f634592"
format_stamp: "Formatted at 2026-06-19 20:32:05 on dist-test-slave-4m58"
server_key: "d816afddbeba5d9faded5a8597ffed3e"
server_key_iv: "fcd487d72e1931473c4e146fff3d6d19"
server_key_version: "encryptionkey@0"
I20260619 20:32:05.073323 9861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:05.099197 9861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:05.099550 9861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:05.099700 9861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:05.103330 9861 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:34433
I20260619 20:32:05.103369 9927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:34433 every 8 connection(s)
I20260619 20:32:05.103693 9861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:32:05.104636 9928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:05.106905 9928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f13bfb2d81448dba5f643920f634592: Bootstrap starting.
I20260619 20:32:05.106935 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 9861
I20260619 20:32:05.107028 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:32:05.107209 4429 external_mini_cluster.cc:1428] Setting key f23c85f7949077b587c770afbdd5c714
I20260619 20:32:05.107470 9928 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f13bfb2d81448dba5f643920f634592: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:05.107797 9928 log.cc:824] T 00000000000000000000000000000000 P 9f13bfb2d81448dba5f643920f634592: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:05.108464 9928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f13bfb2d81448dba5f643920f634592: No bootstrap required, opened a new log
I20260619 20:32:05.110412 9928 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f13bfb2d81448dba5f643920f634592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f13bfb2d81448dba5f643920f634592" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 34433 } }
I20260619 20:32:05.110584 9928 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f13bfb2d81448dba5f643920f634592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:05.110615 9928 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f13bfb2d81448dba5f643920f634592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f13bfb2d81448dba5f643920f634592, State: Initialized, Role: FOLLOWER
I20260619 20:32:05.110747 9928 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f13bfb2d81448dba5f643920f634592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f13bfb2d81448dba5f643920f634592" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 34433 } }
I20260619 20:32:05.110824 9928 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f13bfb2d81448dba5f643920f634592 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:32:05.110847 9928 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f13bfb2d81448dba5f643920f634592 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:32:05.110893 9928 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f13bfb2d81448dba5f643920f634592 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:05.111466 9928 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f13bfb2d81448dba5f643920f634592 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f13bfb2d81448dba5f643920f634592" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 34433 } }
I20260619 20:32:05.111573 9928 leader_election.cc:302] T 00000000000000000000000000000000 P 9f13bfb2d81448dba5f643920f634592 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f13bfb2d81448dba5f643920f634592; no voters:
I20260619 20:32:05.111742 9928 leader_election.cc:288] T 00000000000000000000000000000000 P 9f13bfb2d81448dba5f643920f634592 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:32:05.111783 9933 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f13bfb2d81448dba5f643920f634592 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:32:05.111903 9933 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f13bfb2d81448dba5f643920f634592 [term 1 LEADER]: Becoming Leader. State: Replica: 9f13bfb2d81448dba5f643920f634592, State: Running, Role: LEADER
I20260619 20:32:05.111975 9933 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f13bfb2d81448dba5f643920f634592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f13bfb2d81448dba5f643920f634592" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 34433 } }
I20260619 20:32:05.112003 9928 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f13bfb2d81448dba5f643920f634592 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:32:05.112838 9931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:05.107697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55078 (local address 127.4.83.126:34433)
0619 20:32:05.107912 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:05.107922 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:05.107954 (+ 32us) server_negotiation.cc:407] Connection header received
0619 20:32:05.108406 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:05.108413 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:05.108469 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:05.108644 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:05.108845 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:05.109750 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:05.110789 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:05.111279 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:05.111376 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:05.112080 (+ 704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:05.112084 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:05.112084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:32:05.112311 (+ 227us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:05.112351 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:32:05.112362 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:05.112417 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:05.112596 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 20:32:05.112696 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":73,"threads_started":1}
I20260619 20:32:05.113644 9935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f13bfb2d81448dba5f643920f634592 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f13bfb2d81448dba5f643920f634592" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f13bfb2d81448dba5f643920f634592" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 34433 } } }
I20260619 20:32:05.113536 9934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f13bfb2d81448dba5f643920f634592 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f13bfb2d81448dba5f643920f634592. Latest consensus state: current_term: 1 leader_uuid: "9f13bfb2d81448dba5f643920f634592" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f13bfb2d81448dba5f643920f634592" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 34433 } } }
I20260619 20:32:05.113722 4429 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:32:05.113731 9935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f13bfb2d81448dba5f643920f634592 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:05.113739 9934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f13bfb2d81448dba5f643920f634592 [sys.catalog]: This master's current role is: LEADER
W20260619 20:32:05.114116 9947 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9f13bfb2d81448dba5f643920f634592: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:32:05.114177 9947 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:32:05.114275 9948 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:32:05.114427 9948 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:32:05.115942 9948 catalog_manager.cc:1347] Generated new cluster ID: c5b19aa34efd443f8879275b0b5e9438
I20260619 20:32:05.116001 9948 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:32:05.141853 9948 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:32:05.142683 9948 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:32:05.160386 9948 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f13bfb2d81448dba5f643920f634592: Generated new TSK 0
I20260619 20:32:05.160619 9948 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:32:05.240703 9931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:05.172747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55088 (local address 127.4.83.126:34433)
0619 20:32:05.172807 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:05.172810 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:05.172867 (+ 57us) server_negotiation.cc:407] Connection header received
0619 20:32:05.172923 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:05.172924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:05.172937 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:05.173024 (+ 87us) server_negotiation.cc:226] Negotiated authn=JWT
0619 20:32:05.173128 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:05.173843 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:05.174672 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:05.174978 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:05.175046 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:05.175223 (+ 177us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 20:32:05.240408 (+ 65185us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 20:32:05.240615 (+ 207us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55088: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":36}
W20260619 20:32:05.240787 9931 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55088: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260619 20:32:05.240808 9957 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:05.172646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:34433 (local address 127.0.0.1:55088)
0619 20:32:05.172799 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:05.172809 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:05.172864 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:05.173023 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:05.173024 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:05.173025 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0619 20:32:05.173077 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:05.173079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:05.173939 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:05.173939 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:05.174581 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:05.174583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:05.175065 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:05.175066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:05.175114 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:05.175120 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 20:32:05.240586 (+ 65466us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 20:32:05.240703 (+ 117us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.83.126:34433: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":75,"threads_started":1}
I20260619 20:32:05.241060 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 9861
2026-06-19T20:32:05Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (293 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (89 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (90 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:32:05 dist-test-slave-4m58 krb5kdc[9969](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:32:05 dist-test-slave-4m58 krb5kdc[9969](info): set up 2 sockets
Jun 19 20:32:05 dist-test-slave-4m58 krb5kdc[9969](info): commencing operation
krb5kdc: starting...
W20260619 20:32:06.376855 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.938s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:32:06 dist-test-slave-4m58 krb5kdc[9969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:32:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:32:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
I20260619 20:32:06.502346 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:35091
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:45357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:35091
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:06.581001 9982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:06.581233 9982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:06.581255 9982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:06.582453 9982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:32:06.582495 9982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:06.582508 9982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:06.582520 9982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:32:06.582530 9982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:32:06.584095 9982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:45357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:35091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:35091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:06.584314 9982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:06.584502 9982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:06.589370 9988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:06.589509 9982 server_base.cc:1034] running on GCE node
W20260619 20:32:06.589377 9987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:06.589591 9990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:06.589859 9982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:06.590178 9982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:06.591334 9982 hybrid_clock.cc:648] HybridClock initialized: now 1781901126591325 us; error 31 us; skew 500 ppm
Jun 19 20:32:06 dist-test-slave-4m58 krb5kdc[9969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901126, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:06.593811 9982 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:32:06.594249 9982 webserver.cc:460] Webserver started at http://127.4.83.126:33479/ using document root <none> and password file <none>
I20260619 20:32:06.594456 9982 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:06.594504 9982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:06.594660 9982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:06.595714 9982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "d459fec902c1400cb0d6e0c41a949d6a"
format_stamp: "Formatted at 2026-06-19 20:32:06 on dist-test-slave-4m58"
server_key: "59aa719c0352add5fe53e1d9b11b4125"
server_key_iv: "27a5e013d0273a96cd2dc65913149bc5"
server_key_version: "encryptionkey@0"
I20260619 20:32:06.596095 9982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "d459fec902c1400cb0d6e0c41a949d6a"
format_stamp: "Formatted at 2026-06-19 20:32:06 on dist-test-slave-4m58"
server_key: "59aa719c0352add5fe53e1d9b11b4125"
server_key_iv: "27a5e013d0273a96cd2dc65913149bc5"
server_key_version: "encryptionkey@0"
I20260619 20:32:06.597342 9982 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:06.598040 9997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:06.598194 9982 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:32:06.598266 9982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "d459fec902c1400cb0d6e0c41a949d6a"
format_stamp: "Formatted at 2026-06-19 20:32:06 on dist-test-slave-4m58"
server_key: "59aa719c0352add5fe53e1d9b11b4125"
server_key_iv: "27a5e013d0273a96cd2dc65913149bc5"
server_key_version: "encryptionkey@0"
I20260619 20:32:06.598330 9982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:06.625530 9982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:06.627332 9982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:06.627465 9982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:06.631309 9982 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:35091
I20260619 20:32:06.631358 10049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:35091 every 8 connection(s)
I20260619 20:32:06.631685 9982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:32:06.632505 10050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:06.634775 10050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d459fec902c1400cb0d6e0c41a949d6a: Bootstrap starting.
I20260619 20:32:06.635346 10050 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d459fec902c1400cb0d6e0c41a949d6a: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:06.635604 10050 log.cc:824] T 00000000000000000000000000000000 P d459fec902c1400cb0d6e0c41a949d6a: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:06.636368 10050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d459fec902c1400cb0d6e0c41a949d6a: No bootstrap required, opened a new log
I20260619 20:32:06.637296 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 9982
I20260619 20:32:06.637390 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:32:06.637548 4429 external_mini_cluster.cc:1428] Setting key 73805bb6297887ffd479cbf39b316b0f
I20260619 20:32:06.638242 10050 raft_consensus.cc:348] T 00000000000000000000000000000000 P d459fec902c1400cb0d6e0c41a949d6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d459fec902c1400cb0d6e0c41a949d6a" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35091 } }
I20260619 20:32:06.638430 10050 raft_consensus.cc:374] T 00000000000000000000000000000000 P d459fec902c1400cb0d6e0c41a949d6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:06.638469 10050 raft_consensus.cc:729] T 00000000000000000000000000000000 P d459fec902c1400cb0d6e0c41a949d6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d459fec902c1400cb0d6e0c41a949d6a, State: Initialized, Role: FOLLOWER
I20260619 20:32:06.638602 10050 consensus_queue.cc:260] T 00000000000000000000000000000000 P d459fec902c1400cb0d6e0c41a949d6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d459fec902c1400cb0d6e0c41a949d6a" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35091 } }
I20260619 20:32:06.638688 10050 raft_consensus.cc:388] T 00000000000000000000000000000000 P d459fec902c1400cb0d6e0c41a949d6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:32:06.638767 10050 raft_consensus.cc:482] T 00000000000000000000000000000000 P d459fec902c1400cb0d6e0c41a949d6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:32:06.638819 10050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d459fec902c1400cb0d6e0c41a949d6a [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:06.639941 10050 raft_consensus.cc:504] T 00000000000000000000000000000000 P d459fec902c1400cb0d6e0c41a949d6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d459fec902c1400cb0d6e0c41a949d6a" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35091 } }
I20260619 20:32:06.640054 10050 leader_election.cc:302] T 00000000000000000000000000000000 P d459fec902c1400cb0d6e0c41a949d6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d459fec902c1400cb0d6e0c41a949d6a; no voters:
I20260619 20:32:06.640208 10050 leader_election.cc:288] T 00000000000000000000000000000000 P d459fec902c1400cb0d6e0c41a949d6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:32:06.640259 10055 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d459fec902c1400cb0d6e0c41a949d6a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:32:06.640400 10055 raft_consensus.cc:686] T 00000000000000000000000000000000 P d459fec902c1400cb0d6e0c41a949d6a [term 1 LEADER]: Becoming Leader. State: Replica: d459fec902c1400cb0d6e0c41a949d6a, State: Running, Role: LEADER
I20260619 20:32:06.640455 10050 sys_catalog.cc:564] T 00000000000000000000000000000000 P d459fec902c1400cb0d6e0c41a949d6a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:32:06.640542 10055 consensus_queue.cc:237] T 00000000000000000000000000000000 P d459fec902c1400cb0d6e0c41a949d6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d459fec902c1400cb0d6e0c41a949d6a" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35091 } }
I20260619 20:32:06.642007 10057 sys_catalog.cc:455] T 00000000000000000000000000000000 P d459fec902c1400cb0d6e0c41a949d6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d459fec902c1400cb0d6e0c41a949d6a. Latest consensus state: current_term: 1 leader_uuid: "d459fec902c1400cb0d6e0c41a949d6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d459fec902c1400cb0d6e0c41a949d6a" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35091 } } }
I20260619 20:32:06.642676 10057 sys_catalog.cc:458] T 00000000000000000000000000000000 P d459fec902c1400cb0d6e0c41a949d6a [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:06.642004 10056 sys_catalog.cc:455] T 00000000000000000000000000000000 P d459fec902c1400cb0d6e0c41a949d6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d459fec902c1400cb0d6e0c41a949d6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d459fec902c1400cb0d6e0c41a949d6a" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35091 } } }
W20260619 20:32:06.642361 10070 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d459fec902c1400cb0d6e0c41a949d6a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260619 20:32:06.642854 10056 sys_catalog.cc:458] T 00000000000000000000000000000000 P d459fec902c1400cb0d6e0c41a949d6a [sys.catalog]: This master's current role is: LEADER
W20260619 20:32:06.642870 10070 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:32:06.642936 10072 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:32:06.643110 10072 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:32:06.644593 10072 catalog_manager.cc:1347] Generated new cluster ID: 5df3a57ddac24f4c9c355157ad45e6e7
I20260619 20:32:06.644649 10072 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:32:06 dist-test-slave-4m58 krb5kdc[9969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:32:06.648540 10054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:06.638070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42280 (local address 127.4.83.126:35091)
0619 20:32:06.638306 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:06.638316 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:06.638350 (+ 34us) server_negotiation.cc:407] Connection header received
0619 20:32:06.638464 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:06.638471 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:06.638529 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:06.638674 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:06.639318 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:06.640708 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:06.641686 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:06.642274 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:06.642380 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:06.645570 (+ 3190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:06.645574 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:06.645575 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:06.645587 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:06.647086 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:06.647414 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:06.647418 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:06.647418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:06.647517 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:06.647696 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:06.647696 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:06.647697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:06.647990 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:06.648152 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:06.648232 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 20:32:06.648369 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":139,"threads_started":1}
I20260619 20:32:06.686425 10072 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:32:06.687155 10072 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:32:06.693691 10072 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d459fec902c1400cb0d6e0c41a949d6a: Generated new TSK 0
I20260619 20:32:06.693912 10072 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.83.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
I20260619 20:32:06.722671 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:0
--local_ip_for_outbound_sockets=127.4.83.65
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:35091
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:45357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:06.805114 10076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:06.805359 10076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:06.805397 10076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:06.806670 10076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:06.806720 10076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:06.806774 10076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:32:06.808461 10076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:45357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:35091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:06.808717 10076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:06.808948 10076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:06.814254 10082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:06.814375 10081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:06.814260 10076 server_base.cc:1034] running on GCE node
W20260619 20:32:06.814248 10084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:06.814774 10076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:06.815052 10076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:06.816206 10076 hybrid_clock.cc:648] HybridClock initialized: now 1781901126816187 us; error 37 us; skew 500 ppm
Jun 19 20:32:06 dist-test-slave-4m58 krb5kdc[9969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901126, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:06.818857 10076 init.cc:377] Logged in from keytab as kudu/127.4.83.65@KRBTEST.COM (short username kudu)
I20260619 20:32:06.819345 10076 webserver.cc:460] Webserver started at http://127.4.83.65:35421/ using document root <none> and password file <none>
I20260619 20:32:06.819550 10076 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:06.819595 10076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:06.819715 10076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:06.820823 10076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-0/data/instance:
uuid: "b508f2347f3049fba3ebfff622f95a24"
format_stamp: "Formatted at 2026-06-19 20:32:06 on dist-test-slave-4m58"
server_key: "af110a7022edd5c970f1b171406e3ddd"
server_key_iv: "a9f45de91bada09ae91f7e307eec0e7b"
server_key_version: "encryptionkey@0"
I20260619 20:32:06.821157 10076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance:
uuid: "b508f2347f3049fba3ebfff622f95a24"
format_stamp: "Formatted at 2026-06-19 20:32:06 on dist-test-slave-4m58"
server_key: "af110a7022edd5c970f1b171406e3ddd"
server_key_iv: "a9f45de91bada09ae91f7e307eec0e7b"
server_key_version: "encryptionkey@0"
I20260619 20:32:06.822526 10076 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:32:06.823392 10091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:06.823592 10076 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:06.823678 10076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "b508f2347f3049fba3ebfff622f95a24"
format_stamp: "Formatted at 2026-06-19 20:32:06 on dist-test-slave-4m58"
server_key: "af110a7022edd5c970f1b171406e3ddd"
server_key_iv: "a9f45de91bada09ae91f7e307eec0e7b"
server_key_version: "encryptionkey@0"
I20260619 20:32:06.823779 10076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:06.840550 10076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:06.842890 10076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:06.843040 10076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:06.843271 10076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:06.843585 10076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:06.843617 10076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:06.843647 10076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:06.843669 10076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:06.849648 10076 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:39861
I20260619 20:32:06.849695 10204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:39861 every 8 connection(s)
I20260619 20:32:06.850008 10076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
Jun 19 20:32:06 dist-test-slave-4m58 krb5kdc[9969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901126, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:32:06.858048 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 10076
I20260619 20:32:06.858174 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance
I20260619 20:32:06.858387 4429 external_mini_cluster.cc:1428] Setting key 853b205a08c7ffe35adb9b5b6a4417f7
I20260619 20:32:06.860409 10054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:06.850700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:54347 (local address 127.4.83.126:35091)
0619 20:32:06.850761 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:06.850763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:06.850962 (+ 199us) server_negotiation.cc:407] Connection header received
0619 20:32:06.852156 (+ 1194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:06.852157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:06.852168 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:06.852229 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:06.852974 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:06.853485 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:06.854660 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:06.854870 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:06.854932 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:06.857393 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:06.857396 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:06.857396 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:06.857408 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:06.858966 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:06.859398 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:06.859400 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:06.859400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:06.859458 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:06.859690 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:06.859690 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:06.859691 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:06.859818 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:06.859925 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:06.860219 (+ 294us) server_negotiation.cc:299] Negotiation successful
0619 20:32:06.860312 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 20:32:06.860596 10207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:06.850681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35091 (local address 127.4.83.65:54347)
0619 20:32:06.850841 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:06.850883 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:06.851986 (+ 1103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:06.852342 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:06.852346 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:06.852647 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:06.852855 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:06.852859 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:06.853592 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:06.853593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:06.854482 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:06.854486 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:06.854943 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:06.854944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:06.855049 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:06.855624 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:06.855633 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:06.857261 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:06.859048 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:06.859060 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:06.859064 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:06.859325 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:06.859516 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:06.859516 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:06.859517 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:06.859633 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:06.859914 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:06.859922 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:06.860072 (+ 150us) client_negotiation.cc:769] Sending connection context
0619 20:32:06.860191 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 20:32:06.860377 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":74,"threads_started":1}
I20260619 20:32:06.861109 10205 heartbeater.cc:347] Connected to a master server at 127.4.83.126:35091
I20260619 20:32:06.861217 10205 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:06.861407 10205 heartbeater.cc:511] Master 127.4.83.126:35091 requested a full tablet report, sending...
I20260619 20:32:06.862003 10014 ts_manager.cc:194] Registered new tserver with Master: b508f2347f3049fba3ebfff622f95a24 (127.4.83.65:39861)
I20260619 20:32:06.862630 10014 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.65@KRBTEST.COM'} at 127.4.83.65:54347
WARNING: no policy specified for kudu/127.4.83.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
I20260619 20:32:06.884617 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:0
--local_ip_for_outbound_sockets=127.4.83.66
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:35091
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:45357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:06.965477 10210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:06.965729 10210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:06.965781 10210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:06.967047 10210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:06.967101 10210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:06.967140 10210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:32:06.968837 10210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:45357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:35091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:06.969107 10210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:06.969331 10210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:06.974262 10216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:06.974272 10218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:06.974305 10210 server_base.cc:1034] running on GCE node
W20260619 20:32:06.974272 10215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:06.974722 10210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:06.974942 10210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:06.976176 10210 hybrid_clock.cc:648] HybridClock initialized: now 1781901126976162 us; error 33 us; skew 500 ppm
Jun 19 20:32:06 dist-test-slave-4m58 krb5kdc[9969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901126, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:06.978899 10210 init.cc:377] Logged in from keytab as kudu/127.4.83.66@KRBTEST.COM (short username kudu)
I20260619 20:32:06.979411 10210 webserver.cc:460] Webserver started at http://127.4.83.66:38753/ using document root <none> and password file <none>
I20260619 20:32:06.979619 10210 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:06.979662 10210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:06.979784 10210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:06.980909 10210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-1/data/instance:
uuid: "a958ae0d69544e18a7f0c1b204ce0858"
format_stamp: "Formatted at 2026-06-19 20:32:06 on dist-test-slave-4m58"
server_key: "77b831fdf2ed52ad429b080ceb561ddb"
server_key_iv: "081e7780f899422b218ab3349f4de31a"
server_key_version: "encryptionkey@0"
I20260619 20:32:06.981256 10210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance:
uuid: "a958ae0d69544e18a7f0c1b204ce0858"
format_stamp: "Formatted at 2026-06-19 20:32:06 on dist-test-slave-4m58"
server_key: "77b831fdf2ed52ad429b080ceb561ddb"
server_key_iv: "081e7780f899422b218ab3349f4de31a"
server_key_version: "encryptionkey@0"
I20260619 20:32:06.982613 10210 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:32:06.983425 10225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:06.983599 10210 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:06.983693 10210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "a958ae0d69544e18a7f0c1b204ce0858"
format_stamp: "Formatted at 2026-06-19 20:32:06 on dist-test-slave-4m58"
server_key: "77b831fdf2ed52ad429b080ceb561ddb"
server_key_iv: "081e7780f899422b218ab3349f4de31a"
server_key_version: "encryptionkey@0"
I20260619 20:32:06.983762 10210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:06.999747 10210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:07.002126 10210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:07.002274 10210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:07.002517 10210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:07.002837 10210 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:07.002871 10210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:07.002900 10210 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:07.002920 10210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:07.008883 10210 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:39245
I20260619 20:32:07.008996 10338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:39245 every 8 connection(s)
I20260619 20:32:07.009387 10210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
Jun 19 20:32:07 dist-test-slave-4m58 krb5kdc[9969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901126, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:32:07.019886 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 10210
I20260619 20:32:07.019999 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance
I20260619 20:32:07.020195 4429 external_mini_cluster.cc:1428] Setting key 5d921bd7d8c7788768b12226c17c37f1
I20260619 20:32:07.021338 10054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:07.010186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:48781 (local address 127.4.83.126:35091)
0619 20:32:07.010244 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:07.010246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:07.010409 (+ 163us) server_negotiation.cc:407] Connection header received
0619 20:32:07.011670 (+ 1261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:07.011672 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:07.011686 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:07.011737 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:07.012656 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:07.013364 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:07.014896 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:07.015166 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:07.015255 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:07.017835 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:07.017838 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:07.017838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:07.017847 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:07.019574 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:07.020194 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:07.020196 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:07.020196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:07.020263 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:07.020599 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:07.020600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:07.020600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:07.020745 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:07.020809 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:07.021135 (+ 326us) server_negotiation.cc:299] Negotiation successful
0619 20:32:07.021242 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 20:32:07.021416 10341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:07.010108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35091 (local address 127.4.83.66:48781)
0619 20:32:07.010281 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:07.010327 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:07.011516 (+ 1189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:07.011864 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:07.011869 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:07.012247 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:07.012542 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:07.012549 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:07.013474 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:07.013476 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:07.014780 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:07.014784 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:07.015282 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:07.015283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:07.015442 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:07.016085 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:07.016094 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:07.017730 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:07.019683 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:07.019693 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:07.019696 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:07.020069 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:07.020368 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:07.020370 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:07.020370 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:07.020529 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:07.020808 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:07.020816 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:07.020992 (+ 176us) client_negotiation.cc:769] Sending connection context
0619 20:32:07.021079 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 20:32:07.021230 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
I20260619 20:32:07.021840 10339 heartbeater.cc:347] Connected to a master server at 127.4.83.126:35091
I20260619 20:32:07.021935 10339 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:07.022112 10339 heartbeater.cc:511] Master 127.4.83.126:35091 requested a full tablet report, sending...
I20260619 20:32:07.022480 10014 ts_manager.cc:194] Registered new tserver with Master: a958ae0d69544e18a7f0c1b204ce0858 (127.4.83.66:39245)
I20260619 20:32:07.022981 10014 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.66@KRBTEST.COM'} at 127.4.83.66:48781
WARNING: no policy specified for kudu/127.4.83.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
I20260619 20:32:07.046129 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:0
--local_ip_for_outbound_sockets=127.4.83.67
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:35091
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:45357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:07.133800 10344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:07.134047 10344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:07.134086 10344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:07.135433 10344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:07.135490 10344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:07.135530 10344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:32:07.137519 10344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:45357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:35091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:07.137871 10344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:07.138244 10344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:07.143690 10352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:07.143712 10350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:07.143997 10344 server_base.cc:1034] running on GCE node
W20260619 20:32:07.143700 10349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:07.148284 10344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:07.148625 10344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:07.149889 10344 hybrid_clock.cc:648] HybridClock initialized: now 1781901127149871 us; error 46 us; skew 500 ppm
Jun 19 20:32:07 dist-test-slave-4m58 krb5kdc[9969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901127, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:07.153644 10344 init.cc:377] Logged in from keytab as kudu/127.4.83.67@KRBTEST.COM (short username kudu)
I20260619 20:32:07.154258 10344 webserver.cc:460] Webserver started at http://127.4.83.67:36051/ using document root <none> and password file <none>
I20260619 20:32:07.154492 10344 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:07.154567 10344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:07.154702 10344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:07.156160 10344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-2/data/instance:
uuid: "ca54c461bf84471e8ba059ee0efae8e3"
format_stamp: "Formatted at 2026-06-19 20:32:07 on dist-test-slave-4m58"
server_key: "5ede9ce16b540c182ac4375d0059c73d"
server_key_iv: "bf3422e83d0a19724b75b124e7ad0ef8"
server_key_version: "encryptionkey@0"
I20260619 20:32:07.156525 10344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance:
uuid: "ca54c461bf84471e8ba059ee0efae8e3"
format_stamp: "Formatted at 2026-06-19 20:32:07 on dist-test-slave-4m58"
server_key: "5ede9ce16b540c182ac4375d0059c73d"
server_key_iv: "bf3422e83d0a19724b75b124e7ad0ef8"
server_key_version: "encryptionkey@0"
I20260619 20:32:07.157892 10344 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:32:07.158752 10359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:07.159045 10344 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:07.159183 10344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "ca54c461bf84471e8ba059ee0efae8e3"
format_stamp: "Formatted at 2026-06-19 20:32:07 on dist-test-slave-4m58"
server_key: "5ede9ce16b540c182ac4375d0059c73d"
server_key_iv: "bf3422e83d0a19724b75b124e7ad0ef8"
server_key_version: "encryptionkey@0"
I20260619 20:32:07.159260 10344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:07.170207 10344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:07.173527 10344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:07.173705 10344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:07.173938 10344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:07.174281 10344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:07.174324 10344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:07.174356 10344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:07.174381 10344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:07.182221 10344 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:45441
I20260619 20:32:07.182381 10472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:45441 every 8 connection(s)
I20260619 20:32:07.182726 10344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
Jun 19 20:32:07 dist-test-slave-4m58 krb5kdc[9969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901127, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:32:07.191680 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 10344
I20260619 20:32:07.191788 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance
I20260619 20:32:07.191998 4429 external_mini_cluster.cc:1428] Setting key 74f4b6cb417e263200ee1d772a73ed17
I20260619 20:32:07.193869 10054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:07.183908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:38723 (local address 127.4.83.126:35091)
0619 20:32:07.184022 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:07.184024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:07.184237 (+ 213us) server_negotiation.cc:407] Connection header received
0619 20:32:07.185225 (+ 988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:07.185227 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:07.185240 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:07.185290 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:07.185978 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:07.186478 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:07.187577 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:07.187775 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:07.187856 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:07.190755 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:07.190757 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:07.190757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:07.190766 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:07.192432 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:07.192911 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:07.192912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:07.192912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:07.192977 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:07.193201 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:07.193202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:07.193202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:07.193333 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:07.193390 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:07.193680 (+ 290us) server_negotiation.cc:299] Negotiation successful
0619 20:32:07.193772 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260619 20:32:07.194111 10475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:07.183812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35091 (local address 127.4.83.67:38723)
0619 20:32:07.184066 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:07.184120 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:07.185111 (+ 991us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:07.185367 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:07.185372 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:07.185664 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:07.185883 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:07.185888 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:07.186593 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:07.186594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:07.187474 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:07.187477 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:07.187876 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:07.187877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:07.188015 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:07.188708 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:07.188717 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:07.190591 (+ 1874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:07.192514 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:07.192527 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:07.192529 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:07.192845 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:07.193032 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:07.193033 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:07.193033 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:07.193148 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:07.193388 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:07.193419 (+ 31us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:07.193560 (+ 141us) client_negotiation.cc:769] Sending connection context
0619 20:32:07.193651 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 20:32:07.193910 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":99,"threads_started":1}
I20260619 20:32:07.194528 10473 heartbeater.cc:347] Connected to a master server at 127.4.83.126:35091
I20260619 20:32:07.194630 10473 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:07.194869 10473 heartbeater.cc:511] Master 127.4.83.126:35091 requested a full tablet report, sending...
I20260619 20:32:07.195386 10014 ts_manager.cc:194] Registered new tserver with Master: ca54c461bf84471e8ba059ee0efae8e3 (127.4.83.67:45441)
I20260619 20:32:07.195785 10014 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.67@KRBTEST.COM'} at 127.4.83.67:38723
I20260619 20:32:07.203856 4429 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/test-admin.keytab.
Jun 19 20:32:07 dist-test-slave-4m58 krb5kdc[9969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:07.220692 4429 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260619 20:32:07.222432 10480 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:07.221602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34574 (local address 127.4.83.65:39861)
0619 20:32:07.221756 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:07.221762 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:07.221771 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:32:07.221795 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:07.221797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:07.221811 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:07.221875 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:07.222352 (+ 477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34574: BlockingRecv error: recv got EOF from 127.0.0.1:34574 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":68,"threads_started":1}
W20260619 20:32:07.222512 4429 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/krb5cc' of type FILE
Jun 19 20:32:07 dist-test-slave-4m58 krb5kdc[9969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:07.223538 4429 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 20:32:07 dist-test-slave-4m58 krb5kdc[9969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:07.224440 4429 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 20:32:07 dist-test-slave-4m58 krb5kdc[9969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.65@KRBTEST.COM
I20260619 20:32:07.232723 10480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:07.225090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34580 (local address 127.4.83.65:39861)
0619 20:32:07.225141 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:07.225143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:07.225152 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:32:07.225219 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:07.225238 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:07.225249 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:07.225295 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:07.225678 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:07.226591 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:07.227374 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:07.227797 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:07.227906 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:07.230136 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:07.230146 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:07.230147 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:07.230154 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:07.231678 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:07.232091 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:07.232094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:07.232095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:07.232149 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:07.232286 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:07.232287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:07.232287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:07.232416 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:07.232513 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:07.232578 (+ 65us) server_negotiation.cc:299] Negotiation successful
0619 20:32:07.232642 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260619 20:32:07.234324 10480 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:07.233901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34596 (local address 127.4.83.65:39861)
0619 20:32:07.233935 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:07.233937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:07.233973 (+ 36us) server_negotiation.cc:407] Connection header received
0619 20:32:07.234016 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:07.234016 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:07.234025 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:07.234068 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:07.234282 (+ 214us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34596: BlockingRecv error: recv got EOF from 127.0.0.1:34596 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 20:32:07.234437 4429 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/krb5cc' of type FILE
Jun 19 20:32:07 dist-test-slave-4m58 krb5kdc[9969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:07.235458 4429 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 20:32:07 dist-test-slave-4m58 krb5kdc[9969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:07.236438 4429 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 20:32:07 dist-test-slave-4m58 krb5kdc[9969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.65@KRBTEST.COM
I20260619 20:32:07.243188 10480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:07.236997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34612 (local address 127.4.83.65:39861)
0619 20:32:07.237041 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:07.237043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:07.237070 (+ 27us) server_negotiation.cc:407] Connection header received
0619 20:32:07.237105 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:07.237106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:07.237114 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:07.237153 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:07.237485 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:07.238008 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:07.238683 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:07.238891 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:07.238952 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:07.241048 (+ 2096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:07.241049 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:07.241050 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:07.241055 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:07.242304 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:07.242592 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:07.242592 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:07.242593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:07.242637 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:07.242793 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:07.242794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:07.242794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:07.242881 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:07.242950 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:07.243037 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 20:32:07.243098 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260619 20:32:07.244758 10480 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:07.244253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34618 (local address 127.4.83.65:39861)
0619 20:32:07.244326 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:07.244327 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:07.244335 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:32:07.244390 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:07.244391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:07.244399 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:07.244442 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:07.244709 (+ 267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34618: BlockingRecv error: recv got EOF from 127.0.0.1:34618 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260619 20:32:07.244906 4429 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/krb5cc' of type FILE: Runtime error: Bad format in credentials cache (filename: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901102012824-4429-0/krb5kdc/krb5cc)
Jun 19 20:32:07 dist-test-slave-4m58 krb5kdc[9969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:07.245942 4429 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 20:32:07 dist-test-slave-4m58 krb5kdc[9969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:07.246819 4429 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 20:32:07 dist-test-slave-4m58 krb5kdc[9969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.65@KRBTEST.COM
I20260619 20:32:07.253924 10480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:07.247355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34630 (local address 127.4.83.65:39861)
0619 20:32:07.247405 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:07.247406 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:07.247456 (+ 50us) server_negotiation.cc:407] Connection header received
0619 20:32:07.247518 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:07.247519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:07.247527 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:07.247564 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:07.247984 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:07.248460 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:07.249204 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:07.249400 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:07.249463 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:07.251517 (+ 2054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:07.251518 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:07.251519 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:07.251526 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:07.252825 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:07.253198 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:07.253199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:07.253199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:07.253255 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:07.253467 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:07.253467 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:07.253468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:07.253565 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:07.253640 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:07.253795 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 20:32:07.253856 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 20:32:07.254657 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 10076
I20260619 20:32:07.259094 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 10210
I20260619 20:32:07.263288 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 10344
I20260619 20:32:07.267160 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 9982
2026-06-19T20:32:07Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1846 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:32:07 dist-test-slave-4m58 krb5kdc[10493](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:32:07 dist-test-slave-4m58 krb5kdc[10493](info): set up 2 sockets
Jun 19 20:32:07 dist-test-slave-4m58 krb5kdc[10493](info): commencing operation
krb5kdc: starting...
W20260619 20:32:08.252921 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.967s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:32:08 dist-test-slave-4m58 krb5kdc[10493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:32:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:32:08Z Disabled control of system clock
WARNING: no policy specified for oryx/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.126.keytab.
Entry for principal oryx/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.126.keytab.
I20260619 20:32:08.374856 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:35083
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.126.keytab
--principal=oryx/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:44621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:35083
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:08.457931 10506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:08.458165 10506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:08.458189 10506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:08.459347 10506 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:32:08.459385 10506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:08.459398 10506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:08.459409 10506 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:32:08.459419 10506 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:32:08.460896 10506 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:44621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:35083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.126.keytab
--principal=oryx/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:35083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:08.461110 10506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:08.461292 10506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:08.466321 10511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:08.466321 10514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:08.466308 10512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:08.466377 10506 server_base.cc:1034] running on GCE node
I20260619 20:32:08.466905 10506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:08.467216 10506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:08.468372 10506 hybrid_clock.cc:648] HybridClock initialized: now 1781901128468352 us; error 36 us; skew 500 ppm
Jun 19 20:32:08 dist-test-slave-4m58 krb5kdc[10493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901128, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:08.474108 10506 init.cc:377] Logged in from keytab as oryx/127.4.83.126@KRBTEST.COM (short username oryx)
I20260619 20:32:08.474606 10506 webserver.cc:460] Webserver started at http://127.4.83.126:39761/ using document root <none> and password file <none>
I20260619 20:32:08.474855 10506 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:08.474907 10506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:08.475037 10506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:08.476171 10506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "c8a05ff823034550b48979e189dfdf11"
format_stamp: "Formatted at 2026-06-19 20:32:08 on dist-test-slave-4m58"
server_key: "74f117dc83e6a8085cda864e7abf5651"
server_key_iv: "be9c4400cd8d21a9866650387281a726"
server_key_version: "encryptionkey@0"
I20260619 20:32:08.476495 10506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "c8a05ff823034550b48979e189dfdf11"
format_stamp: "Formatted at 2026-06-19 20:32:08 on dist-test-slave-4m58"
server_key: "74f117dc83e6a8085cda864e7abf5651"
server_key_iv: "be9c4400cd8d21a9866650387281a726"
server_key_version: "encryptionkey@0"
I20260619 20:32:08.477851 10506 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:32:08.478655 10521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:08.478855 10506 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:08.478945 10506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "c8a05ff823034550b48979e189dfdf11"
format_stamp: "Formatted at 2026-06-19 20:32:08 on dist-test-slave-4m58"
server_key: "74f117dc83e6a8085cda864e7abf5651"
server_key_iv: "be9c4400cd8d21a9866650387281a726"
server_key_version: "encryptionkey@0"
I20260619 20:32:08.479010 10506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:08.505600 10506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:08.508015 10506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:08.508188 10506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:08.511904 10506 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:35083
I20260619 20:32:08.512004 10573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:35083 every 8 connection(s)
I20260619 20:32:08.512297 10506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:32:08.513262 10574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:08.515566 10574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11: Bootstrap starting.
I20260619 20:32:08.516180 10574 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:08.516450 10574 log.cc:824] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:08.517155 10574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11: No bootstrap required, opened a new log
I20260619 20:32:08.518774 10574 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } }
I20260619 20:32:08.518924 10574 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:08.518946 10574 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8a05ff823034550b48979e189dfdf11, State: Initialized, Role: FOLLOWER
I20260619 20:32:08.519080 10574 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } }
I20260619 20:32:08.519153 10574 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:32:08.519176 10574 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:32:08.519203 10574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:08.520076 10574 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } }
I20260619 20:32:08.520167 10574 leader_election.cc:302] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8a05ff823034550b48979e189dfdf11; no voters:
I20260619 20:32:08.520324 10574 leader_election.cc:288] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:32:08.520368 10577 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:32:08.520401 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 10506
I20260619 20:32:08.520505 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:32:08.520501 10577 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [term 1 LEADER]: Becoming Leader. State: Replica: c8a05ff823034550b48979e189dfdf11, State: Running, Role: LEADER
I20260619 20:32:08.520576 10577 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } }
I20260619 20:32:08.520601 10574 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:32:08.520705 4429 external_mini_cluster.cc:1428] Setting key 5edb3df6a9cc822276f0ac6450957c7b
I20260619 20:32:08.520864 10577 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8a05ff823034550b48979e189dfdf11. Latest consensus state: current_term: 1 leader_uuid: "c8a05ff823034550b48979e189dfdf11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } }
I20260619 20:32:08.520996 10579 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8a05ff823034550b48979e189dfdf11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } }
I20260619 20:32:08.521014 10577 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:08.521054 10579 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:08.521238 10582 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:32:08.521955 10582 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:32:08.523682 10582 catalog_manager.cc:1347] Generated new cluster ID: 32dbae523902416f99c2f5cbddadd76c
I20260619 20:32:08.523738 10582 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:32:08 dist-test-slave-4m58 krb5kdc[10493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM
I20260619 20:32:08.530339 10582 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:32:08.531044 10582 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:32:08.531278 10588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:08.521343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56504 (local address 127.4.83.126:35083)
0619 20:32:08.521699 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:08.521709 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:08.521747 (+ 38us) server_negotiation.cc:407] Connection header received
0619 20:32:08.521883 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:08.521890 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:08.521952 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:08.522094 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:08.523035 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:08.524424 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:08.525254 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:08.525685 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:08.525801 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:08.528145 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:08.528149 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:08.528150 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:08.528163 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:08.529719 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:08.530082 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:08.530085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:08.530086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:08.530177 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:08.530403 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:08.530404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:08.530404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:08.530686 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:08.530828 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:08.530951 (+ 123us) server_negotiation.cc:299] Negotiation successful
0619 20:32:08.531089 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":67,"threads_started":1}
I20260619 20:32:08.537662 10582 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11: Generated new TSK 0
I20260619 20:32:08.537851 10582 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.4.83.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.83.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.83.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.65.keytab.
Entry for principal oryx/127.4.83.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.65.keytab.
I20260619 20:32:08.567018 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:0
--local_ip_for_outbound_sockets=127.4.83.65
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:35083
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.65.keytab
--principal=oryx/127.4.83.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:44621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:08.661895 10600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:08.662111 10600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:08.662137 10600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:08.663296 10600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:08.663339 10600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:08.663365 10600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:32:08.664841 10600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:44621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.65.keytab
--principal=oryx/127.4.83.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:35083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:08.665062 10600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:08.665283 10600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:08.670785 10608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:08.670791 10606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:08.671041 10600 server_base.cc:1034] running on GCE node
W20260619 20:32:08.670785 10605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:08.671321 10600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:08.671566 10600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:08.672729 10600 hybrid_clock.cc:648] HybridClock initialized: now 1781901128672713 us; error 30 us; skew 500 ppm
Jun 19 20:32:08 dist-test-slave-4m58 krb5kdc[10493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901128, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:08.675276 10600 init.cc:377] Logged in from keytab as oryx/127.4.83.65@KRBTEST.COM (short username oryx)
I20260619 20:32:08.675673 10600 webserver.cc:460] Webserver started at http://127.4.83.65:38353/ using document root <none> and password file <none>
I20260619 20:32:08.675879 10600 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:08.675926 10600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:08.676026 10600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:08.677026 10600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/data/instance:
uuid: "50ef078492ce4fa6a41e0f986ce8390d"
format_stamp: "Formatted at 2026-06-19 20:32:08 on dist-test-slave-4m58"
server_key: "1b977f914dd9e6c468b3920b681b5e83"
server_key_iv: "b15dd9b8a6a728e4829e7e568d7d34e5"
server_key_version: "encryptionkey@0"
I20260619 20:32:08.677310 10600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance:
uuid: "50ef078492ce4fa6a41e0f986ce8390d"
format_stamp: "Formatted at 2026-06-19 20:32:08 on dist-test-slave-4m58"
server_key: "1b977f914dd9e6c468b3920b681b5e83"
server_key_iv: "b15dd9b8a6a728e4829e7e568d7d34e5"
server_key_version: "encryptionkey@0"
I20260619 20:32:08.678467 10600 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 20:32:08.679100 10615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:08.679248 10600 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:32:08.679327 10600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "50ef078492ce4fa6a41e0f986ce8390d"
format_stamp: "Formatted at 2026-06-19 20:32:08 on dist-test-slave-4m58"
server_key: "1b977f914dd9e6c468b3920b681b5e83"
server_key_iv: "b15dd9b8a6a728e4829e7e568d7d34e5"
server_key_version: "encryptionkey@0"
I20260619 20:32:08.679383 10600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:08.704130 10600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:08.706177 10600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:08.706295 10600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:08.706496 10600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:08.706763 10600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:08.706790 10600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:08.706812 10600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:08.706822 10600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:08.712102 10600 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:35511
I20260619 20:32:08.712178 10728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:35511 every 8 connection(s)
I20260619 20:32:08.712463 10600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
I20260619 20:32:08.713121 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 10600
I20260619 20:32:08.713222 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance
I20260619 20:32:08.713421 4429 external_mini_cluster.cc:1428] Setting key 31bd55bb67f3ccee4299b821423174a9
Jun 19 20:32:08 dist-test-slave-4m58 krb5kdc[10493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901128, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM
WARNING: no policy specified for oryx/127.4.83.66@KRBTEST.COM; defaulting to no policy
I20260619 20:32:08.722468 10588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:08.713205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:53959 (local address 127.4.83.126:35083)
0619 20:32:08.713268 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:08.713270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:08.713463 (+ 193us) server_negotiation.cc:407] Connection header received
0619 20:32:08.714044 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:08.714044 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:08.714057 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:08.714095 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:08.715058 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:08.715793 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:08.716871 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:08.717114 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:08.717180 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:08.719512 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:08.719513 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:08.719513 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:08.719521 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:08.720921 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:08.721412 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:08.721413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:08.721414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:08.721468 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:08.721792 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:08.721793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:08.721794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:08.721927 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:08.721997 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:08.722285 (+ 288us) server_negotiation.cc:299] Negotiation successful
0619 20:32:08.722369 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 20:32:08.722708 10731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:08.713163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.4.83.65:53959)
0619 20:32:08.713334 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:08.713374 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:08.713957 (+ 583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:08.714184 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:08.714189 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:08.714754 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:08.714950 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:08.714954 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:08.715940 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:08.715942 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:08.716784 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:08.716787 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:08.717198 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:08.717200 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:08.717328 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:08.717878 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:08.717887 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:08.719389 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:08.721016 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:08.721033 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:08.721035 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:08.721334 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:08.721524 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:08.721525 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:08.721525 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:08.721663 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:08.722019 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:08.722025 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:08.722164 (+ 139us) client_negotiation.cc:769] Sending connection context
0619 20:32:08.722262 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 20:32:08.722411 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":76,"threads_started":1}
I20260619 20:32:08.723203 10729 heartbeater.cc:347] Connected to a master server at 127.4.83.126:35083
I20260619 20:32:08.723304 10729 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:08.723505 10729 heartbeater.cc:511] Master 127.4.83.126:35083 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.83.66@KRBTEST.COM" created.
I20260619 20:32:08.724177 10538 ts_manager.cc:194] Registered new tserver with Master: 50ef078492ce4fa6a41e0f986ce8390d (127.4.83.65:35511)
I20260619 20:32:08.724802 10538 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.83.65@KRBTEST.COM'} at 127.4.83.65:53959
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.83.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.66.keytab.
Entry for principal oryx/127.4.83.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.66.keytab.
I20260619 20:32:08.738809 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:0
--local_ip_for_outbound_sockets=127.4.83.66
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:35083
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.66.keytab
--principal=oryx/127.4.83.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:44621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:08.819028 10734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:08.819253 10734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:08.819278 10734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:08.820487 10734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:08.820533 10734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:08.820560 10734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:32:08.822108 10734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:44621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.66.keytab
--principal=oryx/127.4.83.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:35083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:08.822312 10734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:08.822500 10734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:08.827885 10739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:08.827891 10742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:08.827885 10740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:08.827911 10734 server_base.cc:1034] running on GCE node
I20260619 20:32:08.828322 10734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:08.828586 10734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:08.829761 10734 hybrid_clock.cc:648] HybridClock initialized: now 1781901128829749 us; error 29 us; skew 500 ppm
Jun 19 20:32:08 dist-test-slave-4m58 krb5kdc[10493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901128, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:08.832432 10734 init.cc:377] Logged in from keytab as oryx/127.4.83.66@KRBTEST.COM (short username oryx)
I20260619 20:32:08.832871 10734 webserver.cc:460] Webserver started at http://127.4.83.66:43981/ using document root <none> and password file <none>
I20260619 20:32:08.833091 10734 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:08.833150 10734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:08.833276 10734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:08.834340 10734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/data/instance:
uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b"
format_stamp: "Formatted at 2026-06-19 20:32:08 on dist-test-slave-4m58"
server_key: "11b36dbf68f6c1d2a67c4a29b8cddd82"
server_key_iv: "62d1a737d8fa26b27927e41b61300bbd"
server_key_version: "encryptionkey@0"
I20260619 20:32:08.834686 10734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance:
uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b"
format_stamp: "Formatted at 2026-06-19 20:32:08 on dist-test-slave-4m58"
server_key: "11b36dbf68f6c1d2a67c4a29b8cddd82"
server_key_iv: "62d1a737d8fa26b27927e41b61300bbd"
server_key_version: "encryptionkey@0"
I20260619 20:32:08.836012 10734 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 20:32:08.836812 10749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:08.836984 10734 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:08.837087 10734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b"
format_stamp: "Formatted at 2026-06-19 20:32:08 on dist-test-slave-4m58"
server_key: "11b36dbf68f6c1d2a67c4a29b8cddd82"
server_key_iv: "62d1a737d8fa26b27927e41b61300bbd"
server_key_version: "encryptionkey@0"
I20260619 20:32:08.837150 10734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:08.850914 10734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:08.853019 10734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:08.853174 10734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:08.853433 10734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:08.853749 10734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:08.853780 10734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:08.853811 10734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:08.853827 10734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:08.859371 10734 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:33863
I20260619 20:32:08.859418 10862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:33863 every 8 connection(s)
I20260619 20:32:08.859736 10734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
I20260619 20:32:08.863662 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 10734
I20260619 20:32:08.863786 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance
I20260619 20:32:08.863996 4429 external_mini_cluster.cc:1428] Setting key 3b99479542dcebf88c56600392e7f7a8
Jun 19 20:32:08 dist-test-slave-4m58 krb5kdc[10493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901128, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM
I20260619 20:32:08.869089 10588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:08.860433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:59121 (local address 127.4.83.126:35083)
0619 20:32:08.860488 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:08.860490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:08.860681 (+ 191us) server_negotiation.cc:407] Connection header received
0619 20:32:08.861183 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:08.861184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:08.861195 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:08.861251 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:08.862164 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:08.862698 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:08.863851 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:08.864118 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:08.864203 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:08.866507 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:08.866509 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:08.866509 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:08.866517 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:08.867789 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:08.868174 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:08.868175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:08.868175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:08.868249 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:08.868471 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:08.868472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:08.868472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:08.868587 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:08.868643 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:08.868906 (+ 263us) server_negotiation.cc:299] Negotiation successful
0619 20:32:08.868997 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:32:08.869254 10865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:08.860414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.4.83.66:59121)
0619 20:32:08.860562 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:08.860597 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:08.861075 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:08.861302 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:08.861305 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:08.861860 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:08.862067 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:08.862071 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:08.862782 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:08.862782 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:08.863710 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:08.863713 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:08.864207 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:08.864208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:08.864350 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:08.864882 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:08.864893 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:08.866415 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:08.867878 (+ 1463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:08.867886 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:08.867889 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:08.868108 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:08.868305 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:08.868306 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:08.868306 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:08.868417 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:08.868640 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:08.868647 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:08.868791 (+ 144us) client_negotiation.cc:769] Sending connection context
0619 20:32:08.868880 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 20:32:08.869040 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":84,"threads_started":1}
I20260619 20:32:08.869669 10863 heartbeater.cc:347] Connected to a master server at 127.4.83.126:35083
I20260619 20:32:08.869777 10863 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:08.869997 10863 heartbeater.cc:511] Master 127.4.83.126:35083 requested a full tablet report, sending...
I20260619 20:32:08.870378 10538 ts_manager.cc:194] Registered new tserver with Master: ec3fe6dc2f68409699ccbe96a09f7d8b (127.4.83.66:33863)
I20260619 20:32:08.870748 10538 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.83.66@KRBTEST.COM'} at 127.4.83.66:59121
WARNING: no policy specified for oryx/127.4.83.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.83.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.83.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.67.keytab.
Entry for principal oryx/127.4.83.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.67.keytab.
I20260619 20:32:08.889856 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:0
--local_ip_for_outbound_sockets=127.4.83.67
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:35083
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.67.keytab
--principal=oryx/127.4.83.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:44621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:08.970446 10868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:08.970685 10868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:08.970711 10868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:08.971923 10868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:08.971969 10868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:08.971997 10868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:32:08.973443 10868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:44621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.67.keytab
--principal=oryx/127.4.83.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:35083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:08.973652 10868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:08.973840 10868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:08.978958 10876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:08.979007 10873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:08.978958 10874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:08.979491 10868 server_base.cc:1034] running on GCE node
I20260619 20:32:08.979732 10868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:08.980011 10868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:08.981177 10868 hybrid_clock.cc:648] HybridClock initialized: now 1781901128981160 us; error 32 us; skew 500 ppm
Jun 19 20:32:08 dist-test-slave-4m58 krb5kdc[10493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901128, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:08.984078 10868 init.cc:377] Logged in from keytab as oryx/127.4.83.67@KRBTEST.COM (short username oryx)
I20260619 20:32:08.984565 10868 webserver.cc:460] Webserver started at http://127.4.83.67:39347/ using document root <none> and password file <none>
I20260619 20:32:08.984777 10868 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:08.984826 10868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:08.984957 10868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:08.986090 10868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/data/instance:
uuid: "46234eb18fcf465f96ba823f2d6753d3"
format_stamp: "Formatted at 2026-06-19 20:32:08 on dist-test-slave-4m58"
server_key: "8b077d089681b35c1a94e21cc2ca71de"
server_key_iv: "cf064913d829b1e96495bb283285207f"
server_key_version: "encryptionkey@0"
I20260619 20:32:08.986483 10868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance:
uuid: "46234eb18fcf465f96ba823f2d6753d3"
format_stamp: "Formatted at 2026-06-19 20:32:08 on dist-test-slave-4m58"
server_key: "8b077d089681b35c1a94e21cc2ca71de"
server_key_iv: "cf064913d829b1e96495bb283285207f"
server_key_version: "encryptionkey@0"
I20260619 20:32:08.987854 10868 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:32:08.988651 10883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:08.988811 10868 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:32:08.988896 10868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "46234eb18fcf465f96ba823f2d6753d3"
format_stamp: "Formatted at 2026-06-19 20:32:08 on dist-test-slave-4m58"
server_key: "8b077d089681b35c1a94e21cc2ca71de"
server_key_iv: "cf064913d829b1e96495bb283285207f"
server_key_version: "encryptionkey@0"
I20260619 20:32:08.988977 10868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:09.014648 10868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:09.017580 10868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:09.017747 10868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:09.017992 10868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:09.018304 10868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:09.018338 10868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:09.018368 10868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:09.018392 10868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:09.024607 10868 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:40087
I20260619 20:32:09.024689 10996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:40087 every 8 connection(s)
I20260619 20:32:09.025033 10868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
Jun 19 20:32:09 dist-test-slave-4m58 krb5kdc[10493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901128, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM
I20260619 20:32:09.034372 10999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.025763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.4.83.67:44319)
0619 20:32:09.025937 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.025976 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:09.026459 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:09.026716 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:09.026720 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:09.027341 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:09.027540 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.027545 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.028368 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.028369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.029227 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.029229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.029588 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.029589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.029685 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.030206 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:09.030214 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:09.031598 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:09.033106 (+ 1508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.033113 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.033115 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.033325 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.033481 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.033482 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.033482 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.033569 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.033850 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:09.033854 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:09.033974 (+ 120us) client_negotiation.cc:769] Sending connection context
0619 20:32:09.034056 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 20:32:09.034198 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":68,"threads_started":1}
I20260619 20:32:09.034368 10588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.025826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:44319 (local address 127.4.83.126:35083)
0619 20:32:09.025891 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.025893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.026116 (+ 223us) server_negotiation.cc:407] Connection header received
0619 20:32:09.026568 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.026569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.026581 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.026641 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:09.027685 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.028271 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.029317 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.029519 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.029582 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.031696 (+ 2114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:09.031697 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:09.031698 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:09.031705 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:09.033021 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.033384 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.033385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.033385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.033435 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.033664 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.033665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.033665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.033791 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:09.033854 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.034074 (+ 220us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.034299 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 20:32:09.035043 10997 heartbeater.cc:347] Connected to a master server at 127.4.83.126:35083
I20260619 20:32:09.035130 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 10868
I20260619 20:32:09.035164 10997 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:09.035213 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance
I20260619 20:32:09.035363 10997 heartbeater.cc:511] Master 127.4.83.126:35083 requested a full tablet report, sending...
I20260619 20:32:09.035400 4429 external_mini_cluster.cc:1428] Setting key a12d5722bcab997630bec836e8e05bf4
I20260619 20:32:09.035794 10537 ts_manager.cc:194] Registered new tserver with Master: 46234eb18fcf465f96ba823f2d6753d3 (127.4.83.67:40087)
I20260619 20:32:09.036271 10537 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.83.67@KRBTEST.COM'} at 127.4.83.67:44319
I20260619 20:32:09.036290 4429 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.4.83.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.83.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.83.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.125.keytab.
Entry for principal oryx/127.4.83.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.125.keytab.
I20260619 20:32:09.064519 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.125:44301
--webserver_interface=127.4.83.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.125.keytab
--principal=oryx/127.4.83.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:44621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:35083,127.4.83.125:44301
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:09.147199 11002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:09.147414 11002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:09.147436 11002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:09.148624 11002 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:32:09.148671 11002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:09.148684 11002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:09.148696 11002 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:32:09.148707 11002 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:32:09.150213 11002 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:44621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:35083,127.4.83.125:44301
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.125.keytab
--principal=oryx/127.4.83.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.125:44301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.83.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
W20260619 20:32:09.150336 11002 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.4.83.126:35083,127.4.83.125:44301'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260619 20:32:09.150486 11002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:09.150727 11002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:09.155465 11008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:09.155449 11007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:09.155668 11002 server_base.cc:1034] running on GCE node
W20260619 20:32:09.155457 11010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:09.156000 11002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:09.156320 11002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:09.157488 11002 hybrid_clock.cc:648] HybridClock initialized: now 1781901129157468 us; error 30 us; skew 500 ppm
Jun 19 20:32:09 dist-test-slave-4m58 krb5kdc[10493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901129, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:09.160157 11002 init.cc:377] Logged in from keytab as oryx/127.4.83.125@KRBTEST.COM (short username oryx)
I20260619 20:32:09.160633 11002 webserver.cc:460] Webserver started at http://127.4.83.125:33521/ using document root <none> and password file <none>
I20260619 20:32:09.160881 11002 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:09.160929 11002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:09.161053 11002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:09.162098 11002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-1/data/instance:
uuid: "b75ee363caf445f6844b230fb17c0e39"
format_stamp: "Formatted at 2026-06-19 20:32:09 on dist-test-slave-4m58"
server_key: "044397b318f8248f6890e34cd39bee25"
server_key_iv: "95d206c7316f825b67d657c1ee1b5c34"
server_key_version: "encryptionkey@0"
I20260619 20:32:09.162428 11002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-1/wal/instance:
uuid: "b75ee363caf445f6844b230fb17c0e39"
format_stamp: "Formatted at 2026-06-19 20:32:09 on dist-test-slave-4m58"
server_key: "044397b318f8248f6890e34cd39bee25"
server_key_iv: "95d206c7316f825b67d657c1ee1b5c34"
server_key_version: "encryptionkey@0"
I20260619 20:32:09.163748 11002 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:32:09.164464 11017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:09.164618 11002 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:32:09.164733 11002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-1/wal
uuid: "b75ee363caf445f6844b230fb17c0e39"
format_stamp: "Formatted at 2026-06-19 20:32:09 on dist-test-slave-4m58"
server_key: "044397b318f8248f6890e34cd39bee25"
server_key_iv: "95d206c7316f825b67d657c1ee1b5c34"
server_key_version: "encryptionkey@0"
I20260619 20:32:09.164803 11002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:09.183911 11002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:09.186209 11002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:09.186437 11002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:09.190358 11002 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.125:44301
I20260619 20:32:09.190388 11069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.125:44301 every 8 connection(s)
I20260619 20:32:09.190722 11002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-1/data/info.pb
I20260619 20:32:09.191653 11070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:09.193336 11070 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 20:32:09 dist-test-slave-4m58 krb5kdc[10493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901129, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.125@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM
I20260619 20:32:09.199744 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 11002
I20260619 20:32:09.202960 10588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.194052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56518 (local address 127.4.83.126:35083)
0619 20:32:09.194132 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.194135 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.194257 (+ 122us) server_negotiation.cc:407] Connection header received
0619 20:32:09.194775 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.194776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.194786 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.194845 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:09.195522 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.196094 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.197091 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.197288 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.197353 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.199629 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:09.199630 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:09.199630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:09.199637 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:09.201302 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.201875 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.201876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.201876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.201939 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.202240 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.202241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.202241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.202417 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:09.202506 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.202769 (+ 263us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.202858 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 20:32:09.203059 11072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.193950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.0.0.1:56518)
0619 20:32:09.194126 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.194164 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:09.194669 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:09.194945 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:09.194948 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:09.195208 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:09.195412 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.195416 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.196182 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.196183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.196984 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.196986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.197383 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.197384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.197502 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.198075 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:09.198085 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:09.199476 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:09.201399 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.201406 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.201408 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.201735 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.202006 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.202007 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.202007 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.202151 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.202497 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:09.202502 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:09.202641 (+ 139us) client_negotiation.cc:769] Sending connection context
0619 20:32:09.202729 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 20:32:09.202865 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":73,"threads_started":1}
I20260619 20:32:09.203740 11070 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 20:32:09 dist-test-slave-4m58 krb5kdc[10493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.83.125@KRBTEST.COM
Jun 19 20:32:09 dist-test-slave-4m58 krb5kdc[10493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901129, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.125@KRBTEST.COM for oryx/127.4.83.125@KRBTEST.COM
I20260619 20:32:09.209558 11074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.200264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52436 (local address 127.4.83.125:44301)
0619 20:32:09.200393 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.200398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.200441 (+ 43us) server_negotiation.cc:407] Connection header received
0619 20:32:09.200707 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.200711 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.200723 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.200782 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:09.201481 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.202408 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.203397 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.203792 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.203918 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.206254 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:09.206263 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:09.206263 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:09.206269 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:09.207925 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.208304 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.208307 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.208307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.208363 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.208843 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.208844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.208844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.209033 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:09.209216 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.209370 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.209452 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":72,"threads_started":1}
I20260619 20:32:09.211381 11072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.204051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:44301 (local address 127.0.0.1:52440)
0619 20:32:09.204100 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.204111 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:09.204383 (+ 272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:09.204526 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:09.204527 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:09.204664 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:09.204706 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.204707 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.205364 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.205365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.205895 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.205897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.206340 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.206341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.206413 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.206853 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:09.206858 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:09.208187 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:09.210535 (+ 2348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.210536 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.210536 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.210710 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.210888 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.210889 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.210889 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.210939 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.211176 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:09.211177 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:09.211207 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 20:32:09.211249 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 20:32:09.211321 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 20:32:09.211480 11075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.204084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52440 (local address 127.4.83.125:44301)
0619 20:32:09.204224 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.204225 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.204231 (+ 6us) server_negotiation.cc:407] Connection header received
0619 20:32:09.204447 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.204447 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.204456 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.204513 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:09.204766 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.205284 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.205971 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.206272 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.206335 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.208259 (+ 1924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:09.208260 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:09.208261 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:09.208281 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:09.210452 (+ 2171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.210769 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.210770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.210770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.210833 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.210990 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.210991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.210991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.211113 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:09.211188 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.211275 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.211373 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":122,"thread_start_us":95,"threads_started":1}
I20260619 20:32:09.212610 11070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39: Bootstrap starting.
I20260619 20:32:09.213212 11070 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:09.213438 11070 log.cc:824] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:09.214145 11070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39: No bootstrap required, opened a new log
I20260619 20:32:09.215556 11070 raft_consensus.cc:348] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } }
I20260619 20:32:09.215688 11070 raft_consensus.cc:374] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:09.215721 11070 raft_consensus.cc:729] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b75ee363caf445f6844b230fb17c0e39, State: Initialized, Role: FOLLOWER
I20260619 20:32:09.215875 11070 consensus_queue.cc:260] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } }
I20260619 20:32:09.216137 11077 sys_catalog.cc:455] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } } }
I20260619 20:32:09.216218 11077 sys_catalog.cc:458] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:32:09.216225 11070 sys_catalog.cc:564] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [sys.catalog]: configured and running, proceeding with master startup.
W20260619 20:32:09.217761 11088 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:32:09.217821 11088 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:32:09.219343 11002 master_runner.cc:428] Detected that this master b75ee363caf445f6844b230fb17c0e39 is joining an existing cluster
I20260619 20:32:09.219380 11002 master_runner.cc:432] Initiating AddMaster RPC to add 127.4.83.125:44301
I20260619 20:32:09.226930 11094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.220320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.0.0.1:56526)
0619 20:32:09.220462 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.220473 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:09.220593 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:09.220737 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:09.220738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:09.220936 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:09.221002 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.221004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.222029 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.222030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.222682 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.222684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.223134 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.223135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.223186 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.223668 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:09.223673 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:09.224042 (+ 369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:09.225982 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.225982 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.225983 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.226161 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.226351 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.226352 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.226352 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.226404 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.226626 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:09.226627 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:09.226669 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 20:32:09.226722 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 20:32:09.226823 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":73,"threads_started":1}
I20260619 20:32:09.226922 10588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.220360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56526 (local address 127.4.83.126:35083)
0619 20:32:09.220402 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.220405 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.220526 (+ 121us) server_negotiation.cc:407] Connection header received
0619 20:32:09.220658 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.220660 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.220671 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.220735 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:09.221100 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.221927 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.222760 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.223059 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.223126 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.224175 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:09.224177 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:09.224177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:09.224185 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:09.225892 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.226238 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.226239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.226239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.226299 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.226454 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.226455 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.226455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.226566 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:09.226629 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.226752 (+ 123us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.226822 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:32:09.230950 11095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.220504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:44301 (local address 127.0.0.1:52442)
0619 20:32:09.223955 (+ 3451us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.223967 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:09.224030 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:09.224267 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:09.224268 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:09.224428 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:09.224478 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.224479 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.225212 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.225213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.226960 (+ 1747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.226963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.227471 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.227472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.227530 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.227991 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:09.227996 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:09.228271 (+ 275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:09.229927 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.229927 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.229928 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.230174 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.230343 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.230344 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.230344 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.230395 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.230597 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:09.230598 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:09.230636 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 20:32:09.230717 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 20:32:09.230812 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3427,"thread_start_us":51,"threads_started":1}
I20260619 20:32:09.231012 11075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.220558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52442 (local address 127.4.83.125:44301)
0619 20:32:09.220591 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.220592 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.224032 (+ 3440us) server_negotiation.cc:407] Connection header received
0619 20:32:09.224184 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.224185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.224210 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.224257 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:09.224541 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.225119 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.227062 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.227362 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.227404 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.228347 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:09.228348 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:09.228349 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:09.228355 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:09.229856 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.230241 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.230242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.230242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.230292 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.230442 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.230443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.230443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.230543 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:09.230592 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.230757 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.230878 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":19}
Jun 19 20:32:09 dist-test-slave-4m58 krb5kdc[10493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901128, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.126@KRBTEST.COM for oryx/127.4.83.125@KRBTEST.COM
I20260619 20:32:09.236789 11074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.230013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52454 (local address 127.4.83.125:44301)
0619 20:32:09.230049 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.230050 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.230057 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:32:09.230139 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.230140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.230149 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.230195 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:09.230676 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.231362 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.232273 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.232495 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.232548 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.234436 (+ 1888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:09.234437 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:09.234438 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:09.234444 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:09.235683 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.236070 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.236072 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.236072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.236126 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.236287 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.236288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.236288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.236392 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:09.236443 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.236608 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.236685 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:32:09.236796 11096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.229815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:44301 (local address 127.0.0.1:52454)
0619 20:32:09.229949 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.229994 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:09.230057 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:09.230200 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:09.230203 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:09.230538 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:09.230599 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.230601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.231449 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.231450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.232202 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.232205 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.232558 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.232558 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.232645 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.233072 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:09.233096 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:09.234361 (+ 1265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:09.235757 (+ 1396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.235764 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.235767 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.236008 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.236178 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.236179 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.236179 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.236239 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.236443 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:09.236451 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:09.236502 (+ 51us) client_negotiation.cc:769] Sending connection context
0619 20:32:09.236572 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 20:32:09.236689 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":60,"threads_started":1}
I20260619 20:32:09.237344 10537 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.4.83.125:44301: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b75ee363caf445f6844b230fb17c0e39"
member_type: NON_VOTER
last_known_addr {
host: "127.4.83.125"
port: 44301
}
attrs {
promote: true
}
}
dest_uuid: "c8a05ff823034550b48979e189dfdf11"
cas_config_opid_index: -1
I20260619 20:32:09.237629 10537 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: NON_VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: true } }
I20260619 20:32:09.238030 11098 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER b75ee363caf445f6844b230fb17c0e39 (127.4.83.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: NON_VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: true } } }
I20260619 20:32:09.238376 11044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:09.239058 11098 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.4.83.125:44301
I20260619 20:32:09.239212 11044 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [term 1 FOLLOWER]: Refusing update from remote peer c8a05ff823034550b48979e189dfdf11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260619 20:32:09.239200 11097 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c8a05ff823034550b48979e189dfdf11" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: NON_VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: true } } }
I20260619 20:32:09.239264 11097 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:09.239333 11002 master.cc:537] Master@127.4.83.125:44301 shutting down...
I20260619 20:32:09.239848 11097 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: NON_VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 20:32:09.240506 10524 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260619 20:32:09.242033 11002 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:32:09.242144 11002 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:32:09.242197 11002 tablet_replica.cc:330] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39: stopping tablet replica
I20260619 20:32:09.248349 11096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.248188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:44301 (local address 127.0.0.1:52456)
0619 20:32:09.248261 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.248310 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.125:44301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260619 20:32:09.248461 10524 consensus_peers.cc:487] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 -> Peer b75ee363caf445f6844b230fb17c0e39 (127.4.83.125:44301): Couldn't send request to peer b75ee363caf445f6844b230fb17c0e39. Status: Network error: Client connection negotiation failed: client connection to 127.4.83.125:44301: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 20:32:09.255278 11002 master.cc:552] Master@127.4.83.125:44301 shutdown complete.
I20260619 20:32:09.255344 11002 master_runner.cc:305] Clearing existing system tablet
I20260619 20:32:09.255470 11002 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:09.255517 11002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:09.256321 11002 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:32:09.256897 11102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:09.257069 11002 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:09.257129 11002 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:09.257153 11002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-1/wal
uuid: "b75ee363caf445f6844b230fb17c0e39"
format_stamp: "Formatted at 2026-06-19 20:32:09 on dist-test-slave-4m58"
server_key: "044397b318f8248f6890e34cd39bee25"
server_key_iv: "95d206c7316f825b67d657c1ee1b5c34"
server_key_version: "encryptionkey@0"
I20260619 20:32:09.257951 11002 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:32:09.259433 11002 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 20:32:09.259507 11002 log.cc:1196] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260619 20:32:09.259704 11002 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39: Deleting consensus metadata
I20260619 20:32:09.259820 11002 master_runner.cc:315] Copying system tablet from 127.4.83.126:35083
I20260619 20:32:09.260458 11002 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39: tablet copy: Beginning tablet copy session from remote peer at address 127.4.83.126:35083
I20260619 20:32:09.267227 10588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.260823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56540 (local address 127.4.83.126:35083)
0619 20:32:09.260878 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.260880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.260999 (+ 119us) server_negotiation.cc:407] Connection header received
0619 20:32:09.261064 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.261065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.261094 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.261140 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:09.261506 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.262295 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.263052 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.263356 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.263429 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.264446 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:09.264447 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:09.264448 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:09.264456 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:09.266008 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.266352 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.266353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.266353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.266421 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.266682 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.266683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.266683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.266822 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:09.266891 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.267040 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.267124 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 20:32:09.267238 11108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.260782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.0.0.1:56540)
0619 20:32:09.260929 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.260939 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:09.261000 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:09.261202 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:09.261203 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:09.261354 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:09.261419 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.261421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.262395 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.262396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.262970 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.262973 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.263466 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.263467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.263525 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.264054 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:09.264059 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:09.264359 (+ 300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:09.266105 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.266106 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.266107 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.266285 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.266512 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.266513 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.266514 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.266617 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.266889 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:09.266890 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:09.266935 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 20:32:09.267003 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 20:32:09.267139 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
I20260619 20:32:09.267602 10558 tablet_copy_service.cc:140] P c8a05ff823034550b48979e189dfdf11: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b75ee363caf445f6844b230fb17c0e39 ({username='oryx', principal='oryx/127.4.83.125@KRBTEST.COM'} at 127.0.0.1:56540)
I20260619 20:32:09.267724 10558 tablet_copy_service.cc:161] P c8a05ff823034550b48979e189dfdf11: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b75ee363caf445f6844b230fb17c0e39 at {username='oryx', principal='oryx/127.4.83.125@KRBTEST.COM'} at 127.0.0.1:56540: session id = b75ee363caf445f6844b230fb17c0e39-00000000000000000000000000000000
I20260619 20:32:09.268383 10558 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 20:32:09.269032 11002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 20:32:09.270327 11002 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39: tablet copy: Starting download of 0 data blocks...
I20260619 20:32:09.270485 11002 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39: tablet copy: Starting download of 1 WAL segments...
I20260619 20:32:09.271415 11002 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 20:32:09.272475 10558 tablet_copy_service.cc:342] P c8a05ff823034550b48979e189dfdf11: Request end of tablet copy session b75ee363caf445f6844b230fb17c0e39-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.4.83.125@KRBTEST.COM'} at 127.0.0.1:56540
I20260619 20:32:09.272540 10558 tablet_copy_service.cc:433] P c8a05ff823034550b48979e189dfdf11: ending tablet copy session b75ee363caf445f6844b230fb17c0e39-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b75ee363caf445f6844b230fb17c0e39
I20260619 20:32:09.275350 11002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:09.276965 11112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:09.277151 11002 server_base.cc:1034] running on GCE node
W20260619 20:32:09.276903 11110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:09.277230 11109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:09.277534 11002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:09.277724 11002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:09.278859 11002 hybrid_clock.cc:648] HybridClock initialized: now 1781901129278840 us; error 39 us; skew 500 ppm
Jun 19 20:32:09 dist-test-slave-4m58 krb5kdc[10493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901129, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:09.280372 11002 init.cc:377] Logged in from keytab as oryx/127.4.83.125@KRBTEST.COM (short username oryx)
I20260619 20:32:09.280634 11002 webserver.cc:460] Webserver started at http://127.4.83.125:41129/ using document root <none> and password file <none>
I20260619 20:32:09.280756 11002 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:09.280787 11002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:09.281389 11002 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:32:09.281996 11119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:09.282140 11002 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:09.282186 11002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-1/wal
uuid: "b75ee363caf445f6844b230fb17c0e39"
format_stamp: "Formatted at 2026-06-19 20:32:09 on dist-test-slave-4m58"
server_key: "044397b318f8248f6890e34cd39bee25"
server_key_iv: "95d206c7316f825b67d657c1ee1b5c34"
server_key_version: "encryptionkey@0"
I20260619 20:32:09.282248 11002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:09.286469 11002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:09.288089 11002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:09.290963 11002 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.125:44301
I20260619 20:32:09.291338 11171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.125:44301 every 8 connection(s)
I20260619 20:32:09.291369 11002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-1/data/info.pb
I20260619 20:32:09.292225 11172 sys_catalog.cc:263] Verifying existing consensus state
I20260619 20:32:09.292620 11172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39: Bootstrap starting.
I20260619 20:32:09.295660 11172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:32:09.296047 11172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39: Bootstrap complete.
I20260619 20:32:09.296340 11172 raft_consensus.cc:348] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: NON_VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: true } }
I20260619 20:32:09.296422 11172 raft_consensus.cc:729] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b75ee363caf445f6844b230fb17c0e39, State: Initialized, Role: LEARNER
I20260619 20:32:09.296459 11172 consensus_queue.cc:260] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: NON_VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: true } }
I20260619 20:32:09.296674 11172 sys_catalog.cc:564] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:32:09.296675 11175 sys_catalog.cc:455] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: NON_VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: true } } }
I20260619 20:32:09.296828 11175 sys_catalog.cc:458] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [sys.catalog]: This master's current role is: LEARNER
I20260619 20:32:09.298174 11186 catalog_manager.cc:1259] Loaded cluster ID: 32dbae523902416f99c2f5cbddadd76c
I20260619 20:32:09.298223 11186 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39: loading cluster ID for follower catalog manager: success
I20260619 20:32:09.299185 11186 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39: acquiring CA information for follower catalog manager: success
I20260619 20:32:09.299476 11186 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 20:32:09.318111 11187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.311694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52474 (local address 127.4.83.125:44301)
0619 20:32:09.311855 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.311857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.311865 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:32:09.311891 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.311892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.311904 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.311964 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:09.312514 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.313204 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.313894 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.314092 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.314180 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.315342 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:09.315343 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:09.315343 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:09.315367 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:09.316897 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.317205 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.317206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.317206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.317262 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.317430 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.317431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.317431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.317544 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:09.317630 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.317743 (+ 113us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.317821 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":70,"threads_started":1}
WARNING: no policy specified for oryx/127.4.83.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.83.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.83.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.124.keytab.
Entry for principal oryx/127.4.83.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.124.keytab.
I20260619 20:32:09.345525 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.124:45667
--webserver_interface=127.4.83.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.124.keytab
--principal=oryx/127.4.83.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:44621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:35083,127.4.83.125:44301,127.4.83.124:45667
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:09.429558 11190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:09.429800 11190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:09.429927 11190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:09.431375 11190 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:32:09.431442 11190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:09.431455 11190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:09.431468 11190 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:32:09.431478 11190 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:32:09.433295 11190 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:44621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:35083,127.4.83.125:44301,127.4.83.124:45667
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.124.keytab
--principal=oryx/127.4.83.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.124:45667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.83.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:09.433584 11190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:09.433842 11190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:09.438696 11196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:09.438689 11198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:09.438896 11190 server_base.cc:1034] running on GCE node
W20260619 20:32:09.438689 11195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:09.439271 11190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:09.439568 11190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:09.440706 11190 hybrid_clock.cc:648] HybridClock initialized: now 1781901129440688 us; error 35 us; skew 500 ppm
Jun 19 20:32:09 dist-test-slave-4m58 krb5kdc[10493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901129, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:09.443302 11190 init.cc:377] Logged in from keytab as oryx/127.4.83.124@KRBTEST.COM (short username oryx)
I20260619 20:32:09.443822 11190 webserver.cc:460] Webserver started at http://127.4.83.124:41283/ using document root <none> and password file <none>
I20260619 20:32:09.444074 11190 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:09.444124 11190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:09.444250 11190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:09.445284 11190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-2/data/instance:
uuid: "3ee0d6ea97c54966bf13b731b01bbd5f"
format_stamp: "Formatted at 2026-06-19 20:32:09 on dist-test-slave-4m58"
server_key: "8eef77044cad67691795cbc57edc0d5e"
server_key_iv: "bdc8f906d54b8daa6a0a3035e648e385"
server_key_version: "encryptionkey@0"
I20260619 20:32:09.445617 11190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-2/wal/instance:
uuid: "3ee0d6ea97c54966bf13b731b01bbd5f"
format_stamp: "Formatted at 2026-06-19 20:32:09 on dist-test-slave-4m58"
server_key: "8eef77044cad67691795cbc57edc0d5e"
server_key_iv: "bdc8f906d54b8daa6a0a3035e648e385"
server_key_version: "encryptionkey@0"
I20260619 20:32:09.446856 11190 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:32:09.447540 11205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:09.447695 11190 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:32:09.447777 11190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-2/wal
uuid: "3ee0d6ea97c54966bf13b731b01bbd5f"
format_stamp: "Formatted at 2026-06-19 20:32:09 on dist-test-slave-4m58"
server_key: "8eef77044cad67691795cbc57edc0d5e"
server_key_iv: "bdc8f906d54b8daa6a0a3035e648e385"
server_key_version: "encryptionkey@0"
I20260619 20:32:09.447881 11190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:09.468703 11190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:09.470922 11190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:09.471072 11190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:09.474917 11190 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.124:45667
I20260619 20:32:09.474970 11257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.124:45667 every 8 connection(s)
I20260619 20:32:09.475301 11190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-2/data/info.pb
I20260619 20:32:09.476351 11258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:09.477888 11258 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:32:09.481352 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 11190
Jun 19 20:32:09 dist-test-slave-4m58 krb5kdc[10493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901129, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.124@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM
Jun 19 20:32:09 dist-test-slave-4m58 krb5kdc[10493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.83.124@KRBTEST.COM
I20260619 20:32:09.488461 10588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.478787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56542 (local address 127.4.83.126:35083)
0619 20:32:09.478853 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.478855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.478993 (+ 138us) server_negotiation.cc:407] Connection header received
0619 20:32:09.479469 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.479470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.479481 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.479538 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:09.480330 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.480833 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.482247 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.482528 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.482591 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.484876 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:09.484879 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:09.484879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:09.484888 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:09.486780 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.487286 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.487288 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.487288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.487357 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.487720 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.487721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.487721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.487898 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:09.487978 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.488272 (+ 294us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.488364 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 20:32:09.488679 11260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.478580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.0.0.1:56542)
0619 20:32:09.478835 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.478875 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:09.479365 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:09.479700 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:09.479704 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:09.480032 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:09.480238 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.480242 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.480936 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.480937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.482130 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.482133 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.482644 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.482644 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.482784 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.483326 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:09.483336 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:09.484789 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:09.486881 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.486889 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.486891 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.487203 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.487441 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.487442 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.487443 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.487619 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.487981 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:09.487988 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:09.488144 (+ 156us) client_negotiation.cc:769] Sending connection context
0619 20:32:09.488243 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 20:32:09.488408 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":121,"threads_started":1}
I20260619 20:32:09.489264 11258 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:32:09.491215 11261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.481862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49026 (local address 127.4.83.124:45667)
0619 20:32:09.482032 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.482035 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.482055 (+ 20us) server_negotiation.cc:407] Connection header received
0619 20:32:09.482086 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.482087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.482122 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.482197 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:09.482770 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.483634 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.484467 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.484838 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.484943 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.487446 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:09.487456 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:09.487456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:09.487463 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:09.489338 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.489763 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.489765 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.489765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.489818 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.490067 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.490067 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.490068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.490207 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:09.490319 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.491059 (+ 740us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.491131 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":107,"threads_started":1}
Jun 19 20:32:09 dist-test-slave-4m58 krb5kdc[10493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901129, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.124@KRBTEST.COM for oryx/127.4.83.125@KRBTEST.COM
I20260619 20:32:09.496665 11260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.489528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:44301 (local address 127.0.0.1:52486)
0619 20:32:09.489770 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.489779 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:09.489885 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:09.490000 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:09.490001 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:09.490147 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:09.490206 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.490208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.490854 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.490855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.491595 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.491598 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.492026 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.492027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.492099 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.492491 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:09.492495 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:09.493974 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:09.495646 (+ 1672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.495647 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.495647 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.495821 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.496034 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.496035 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.496036 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.496101 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.496331 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:09.496332 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:09.496382 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 20:32:09.496442 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 20:32:09.496560 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229}
I20260619 20:32:09.496701 11187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.489589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52486 (local address 127.4.83.125:44301)
0619 20:32:09.489640 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.489642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.489888 (+ 246us) server_negotiation.cc:407] Connection header received
0619 20:32:09.489944 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.489945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.489957 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.489996 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:09.490278 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.490782 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.491684 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.491964 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.492023 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.494088 (+ 2065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:09.494090 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:09.494090 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:09.494099 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:09.495485 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.495916 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.495917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.495917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.495978 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.496152 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.496153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.496153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.496275 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:09.496331 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.496471 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.496620 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:32:09.497212 11258 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.124" port: 45667 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 20:32:09 dist-test-slave-4m58 krb5kdc[10493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901129, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.124@KRBTEST.COM for oryx/127.4.83.124@KRBTEST.COM
I20260619 20:32:09.503569 11261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.497545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49036 (local address 127.4.83.124:45667)
0619 20:32:09.497578 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.497580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.497652 (+ 72us) server_negotiation.cc:407] Connection header received
0619 20:32:09.497718 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.497718 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.497731 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.497782 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:09.498021 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.498535 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.499191 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.499402 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.499463 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.501331 (+ 1868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:09.501332 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:09.501332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:09.501339 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:09.502580 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.502865 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.502866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.502866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.502910 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.503059 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.503060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.503060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.503173 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:09.503225 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.503345 (+ 120us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.503480 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":15}
I20260619 20:32:09.503569 11260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.497513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:45667 (local address 127.0.0.1:49036)
0619 20:32:09.497601 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.497611 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:09.497660 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:09.497795 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:09.497795 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:09.497920 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:09.497958 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.497960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.498593 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.498593 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.499075 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.499076 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.499485 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.499486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.499530 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.500015 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:09.500019 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:09.501248 (+ 1229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:09.502655 (+ 1407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.502656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.502656 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.502806 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.502959 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.502960 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.502960 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.503011 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.503225 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:09.503225 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:09.503262 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 20:32:09.503322 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 20:32:09.503433 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71,"mutex_wait_us":1}
I20260619 20:32:09.505106 11258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f: Bootstrap starting.
I20260619 20:32:09.505980 11258 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:09.506251 11258 log.cc:824] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:09.507038 11258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f: No bootstrap required, opened a new log
I20260619 20:32:09.508612 11258 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 45667 } }
I20260619 20:32:09.508744 11258 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:09.508766 11258 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ee0d6ea97c54966bf13b731b01bbd5f, State: Initialized, Role: FOLLOWER
I20260619 20:32:09.508911 11258 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 45667 } }
I20260619 20:32:09.509305 11258 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:32:09.509275 11263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 45667 } } }
I20260619 20:32:09.509366 11263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [sys.catalog]: This master's current role is: FOLLOWER
W20260619 20:32:09.511044 11274 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:32:09.511109 11274 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:32:09.513469 11190 master_runner.cc:428] Detected that this master 3ee0d6ea97c54966bf13b731b01bbd5f is joining an existing cluster
I20260619 20:32:09.513520 11190 master_runner.cc:432] Initiating AddMaster RPC to add 127.4.83.124:45667
I20260619 20:32:09.522740 11261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.514952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49038 (local address 127.4.83.124:45667)
0619 20:32:09.514991 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.514992 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.515228 (+ 236us) server_negotiation.cc:407] Connection header received
0619 20:32:09.515316 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.515317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.515325 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.515349 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:09.515885 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.516609 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.517602 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.517902 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.517939 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.520031 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:09.520033 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:09.520033 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:09.520039 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:09.521582 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.522025 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.522026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.522026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.522081 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.522270 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.522271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.522271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.522385 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:09.522446 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.522597 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.522670 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:32:09.522908 11281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.515039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:45667 (local address 127.0.0.1:49038)
0619 20:32:09.515171 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.515180 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:09.515257 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:09.515396 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:09.515397 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:09.515698 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:09.515759 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.515762 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.516704 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.516705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.517510 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.517513 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.518007 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.518007 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.518067 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.519392 (+ 1325us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:09.519398 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:09.519945 (+ 547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:09.521702 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.521703 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.521703 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.521958 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.522146 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.522147 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.522147 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.522213 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.522446 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:09.522447 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:09.522504 (+ 57us) client_negotiation.cc:769] Sending connection context
0619 20:32:09.522579 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 20:32:09.522751 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":58,"thread_start_us":83,"threads_started":1}
I20260619 20:32:09.542975 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 10600
I20260619 20:32:09.544117 10588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.514572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56548 (local address 127.4.83.126:35083)
0619 20:32:09.514617 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.514619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.514759 (+ 140us) server_negotiation.cc:407] Connection header received
0619 20:32:09.514991 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.514992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.515005 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.515035 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:09.515523 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.516403 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.517559 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.517861 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.517901 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.541293 (+ 23392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:09.541295 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:09.541296 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:09.541304 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:09.542838 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.543242 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.543243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.543244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.543303 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.543488 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.543489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.543489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.543634 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:09.543680 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.543921 (+ 241us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.544011 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:32:09.544428 11280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.514531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.0.0.1:56548)
0619 20:32:09.514700 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.514711 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:09.514784 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:09.515114 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:09.515116 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:09.515359 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:09.515441 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.515444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.516496 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.516497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.517459 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.517461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.517982 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.517983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.518045 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.540467 (+ 22422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:09.540473 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:09.541139 (+ 666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:09.542926 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.542927 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.542927 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.543159 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.543368 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.543368 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.543369 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.543421 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.543725 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:09.543726 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:09.543775 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 20:32:09.544200 (+ 425us) client_negotiation.cc:240] Negotiation successful
0619 20:32:09.544321 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":4,"thread_start_us":120,"threads_started":1}
I20260619 20:32:09.544646 11187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.515049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52490 (local address 127.4.83.125:44301)
0619 20:32:09.515092 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.515093 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.515218 (+ 125us) server_negotiation.cc:407] Connection header received
0619 20:32:09.515314 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.515314 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.515323 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.515351 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:09.515797 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.516530 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.517559 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.517863 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.517901 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.541263 (+ 23362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:09.541265 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:09.541266 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:09.541275 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:09.543016 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.543797 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.543798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.543798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.543890 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.544089 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.544089 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.544090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.544217 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:09.544258 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.544449 (+ 191us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.544537 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:32:09.544948 11282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.514772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:44301 (local address 127.0.0.1:52490)
0619 20:32:09.515164 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.515173 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:09.515249 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:09.515411 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:09.515412 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:09.515633 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:09.515708 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.515710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.516610 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.516611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.517462 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.517464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.517991 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.517992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.518047 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.540597 (+ 22550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:09.540601 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:09.541129 (+ 528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:09.543467 (+ 2338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.543468 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.543468 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.543701 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.543964 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.543965 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.543965 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.544026 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.544304 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:09.544305 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:09.544347 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 20:32:09.544732 (+ 385us) client_negotiation.cc:240] Negotiation successful
0619 20:32:09.544840 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":1,"thread_start_us":156,"threads_started":1}
I20260619 20:32:09.549607 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:35511
--local_ip_for_outbound_sockets=127.4.83.65
--tserver_master_addrs=127.4.83.126:35083
--webserver_port=38353
--webserver_interface=127.4.83.65
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.65.keytab
--principal=oryx/127.4.83.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:44621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.83.126:35083,127.4.83.125:44301
--tserver_master_addrs=127.4.83.126:35083,127.4.83.125:44301,127.4.83.124:45667 with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 20:32:09 dist-test-slave-4m58 krb5kdc[10493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901128, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.126@KRBTEST.COM for oryx/127.4.83.124@KRBTEST.COM
I20260619 20:32:09.554809 11096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.547676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:45667 (local address 127.0.0.1:49048)
0619 20:32:09.547963 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.547972 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:09.548018 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:09.548209 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:09.548210 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:09.548383 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:09.548434 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.548436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.549208 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.549209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.549897 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.549900 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.550400 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.550401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.550465 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.550907 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:09.550913 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:09.552410 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:09.553838 (+ 1428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.553839 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.553840 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.554012 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.554187 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.554188 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.554188 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.554247 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.554461 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:09.554462 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:09.554531 (+ 69us) client_negotiation.cc:769] Sending connection context
0619 20:32:09.554590 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 20:32:09.554709 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272}
I20260619 20:32:09.554809 11261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.547732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49048 (local address 127.4.83.124:45667)
0619 20:32:09.547770 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.547772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.548017 (+ 245us) server_negotiation.cc:407] Connection header received
0619 20:32:09.548107 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.548108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.548118 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.548160 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:09.548505 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.549149 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.549991 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.550277 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.550329 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.552490 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:09.552492 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:09.552493 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:09.552500 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:09.553752 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.554076 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.554078 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.554078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.554134 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.554294 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.554295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.554295 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.554410 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:09.554465 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.554631 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.554707 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 20:32:09.555311 10537 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.4.83.124:45667: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f"
member_type: NON_VOTER
last_known_addr {
host: "127.4.83.124"
port: 45667
}
attrs {
promote: true
}
}
dest_uuid: "c8a05ff823034550b48979e189dfdf11"
cas_config_opid_index: 5
I20260619 20:32:09.555563 10537 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: NON_VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: true } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: NON_VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: true } }
I20260619 20:32:09.555920 11098 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 3ee0d6ea97c54966bf13b731b01bbd5f (127.4.83.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: NON_VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: true } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: NON_VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: true } } }
I20260619 20:32:09.556308 11232 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:09.556846 11098 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.4.83.124:45667
I20260619 20:32:09.556919 11098 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c8a05ff823034550b48979e189dfdf11" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: NON_VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: true } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: NON_VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: true } } }
I20260619 20:32:09.556979 11098 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:09.557282 11232 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [term 1 FOLLOWER]: Refusing update from remote peer c8a05ff823034550b48979e189dfdf11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 20:32:09.557785 11190 master.cc:537] Master@127.4.83.124:45667 shutting down...
I20260619 20:32:09.558120 11098 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: NON_VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 20:32:09.558866 10523 consensus_peers.cc:487] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 -> Peer 3ee0d6ea97c54966bf13b731b01bbd5f (127.4.83.124:45667): Couldn't send request to peer 3ee0d6ea97c54966bf13b731b01bbd5f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 20:32:09.561719 11096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.557172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:44301 (local address 127.0.0.1:52506)
0619 20:32:09.557272 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.557282 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:09.557322 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:09.557620 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:09.557621 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:09.557623 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:09.557669 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.557671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.559642 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.559643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.560538 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.560542 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.561428 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.561429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.561505 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.561544 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 20:32:09.561592 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 20:32:09.561642 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260619 20:32:09.562177 11187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.557399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52506 (local address 127.4.83.125:44301)
0619 20:32:09.557440 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.557442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.557449 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:32:09.557475 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.557475 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.557498 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.557550 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:09.557741 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.558453 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.560655 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.561233 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.561306 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.561392 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.562052 (+ 660us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.562112 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:32:09.562489 11146 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [term 1 LEARNER]: Refusing update from remote peer c8a05ff823034550b48979e189dfdf11: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 20:32:09.562731 11190 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:32:09.562835 11190 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:32:09.562888 11190 tablet_replica.cc:330] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f: stopping tablet replica
I20260619 20:32:09.563366 11284 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: NON_VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260619 20:32:09.564338 11146 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 3ee0d6ea97c54966bf13b731b01bbd5f (127.4.83.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: NON_VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: true } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: NON_VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: true } } }
I20260619 20:32:09.565457 11286 sys_catalog.cc:455] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c8a05ff823034550b48979e189dfdf11" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: NON_VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: true } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: NON_VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: true } } }
I20260619 20:32:09.565542 11286 sys_catalog.cc:458] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [sys.catalog]: This master's current role is: LEARNER
I20260619 20:32:09.565675 11175 sys_catalog.cc:455] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8a05ff823034550b48979e189dfdf11. Latest consensus state: current_term: 1 leader_uuid: "c8a05ff823034550b48979e189dfdf11" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: NON_VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: true } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: NON_VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: true } } }
I20260619 20:32:09.565718 11175 sys_catalog.cc:458] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [sys.catalog]: This master's current role is: LEARNER
I20260619 20:32:09.568377 11284 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11: attempting to promote NON_VOTER b75ee363caf445f6844b230fb17c0e39 to VOTER
I20260619 20:32:09.568655 11284 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: false } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: NON_VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: true } }
I20260619 20:32:09.569167 11096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.569046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:45667 (local address 127.0.0.1:49058)
0619 20:32:09.569098 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.569135 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:45667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 20:32:09.569263 11146 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [term 1 LEARNER]: Refusing update from remote peer c8a05ff823034550b48979e189dfdf11: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260619 20:32:09.569536 11098 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260619 20:32:09.569576 11096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.569496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:45667 (local address 127.0.0.1:49074)
0619 20:32:09.569531 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.569554 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:45667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260619 20:32:09.569658 10523 consensus_peers.cc:487] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 -> Peer 3ee0d6ea97c54966bf13b731b01bbd5f (127.4.83.124:45667): Couldn't send request to peer 3ee0d6ea97c54966bf13b731b01bbd5f. Status: Network error: Client connection negotiation failed: client connection to 127.4.83.124:45667: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 20:32:09.570297 11146 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, b75ee363caf445f6844b230fb17c0e39 (127.4.83.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: false } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: NON_VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: true } } }
I20260619 20:32:09.571038 11175 sys_catalog.cc:455] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c8a05ff823034550b48979e189dfdf11" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: false } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: NON_VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: true } } }
I20260619 20:32:09.571120 11175 sys_catalog.cc:458] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:32:09.570204 11284 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, b75ee363caf445f6844b230fb17c0e39 (127.4.83.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: false } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: NON_VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: true } } }
I20260619 20:32:09.572170 11098 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c8a05ff823034550b48979e189dfdf11" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: false } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: NON_VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: true } } }
I20260619 20:32:09.572252 11098 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:09.572333 11284 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c8a05ff823034550b48979e189dfdf11" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: false } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: NON_VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: true } } }
I20260619 20:32:09.572384 11284 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:09.575987 11190 master.cc:552] Master@127.4.83.124:45667 shutdown complete.
I20260619 20:32:09.576066 11190 master_runner.cc:305] Clearing existing system tablet
I20260619 20:32:09.576195 11190 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:09.576237 11190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:09.576803 11190 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:32:09.577242 11288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:09.577378 11190 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:09.577425 11190 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:09.577446 11190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-2/wal
uuid: "3ee0d6ea97c54966bf13b731b01bbd5f"
format_stamp: "Formatted at 2026-06-19 20:32:09 on dist-test-slave-4m58"
server_key: "8eef77044cad67691795cbc57edc0d5e"
server_key_iv: "bdc8f906d54b8daa6a0a3035e648e385"
server_key_version: "encryptionkey@0"
I20260619 20:32:09.578043 11190 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:32:09.579322 11190 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 20:32:09.579391 11190 log.cc:1196] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260619 20:32:09.579598 11190 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f: Deleting consensus metadata
I20260619 20:32:09.579705 11190 master_runner.cc:315] Copying system tablet from 127.4.83.126:35083
I20260619 20:32:09.580344 11190 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f: tablet copy: Beginning tablet copy session from remote peer at address 127.4.83.126:35083
I20260619 20:32:09.586730 10588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.580727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56564 (local address 127.4.83.126:35083)
0619 20:32:09.580780 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.580782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.580836 (+ 54us) server_negotiation.cc:407] Connection header received
0619 20:32:09.580914 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.580915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.580925 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.580979 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:09.581280 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.581855 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.582705 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.582948 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.583016 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.584357 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:09.584358 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:09.584358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:09.584365 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:09.585682 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.586034 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.586035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.586036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.586095 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.586250 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.586251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.586251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.586365 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:09.586420 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.586557 (+ 137us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.586636 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 20:32:09.586747 11294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.580658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.0.0.1:56564)
0619 20:32:09.580783 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.580794 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:09.580852 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:09.580988 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:09.580988 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:09.581155 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:09.581204 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.581206 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.581937 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.581938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.582629 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.582632 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.583016 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.583017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.583079 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.583986 (+ 907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:09.583990 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:09.584273 (+ 283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:09.585756 (+ 1483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.585757 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.585758 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.585966 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.586143 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.586144 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.586144 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.586201 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.586417 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:09.586418 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:09.586463 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 20:32:09.586523 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 20:32:09.586635 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":70,"threads_started":1}
I20260619 20:32:09.587267 10558 tablet_copy_service.cc:140] P c8a05ff823034550b48979e189dfdf11: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 3ee0d6ea97c54966bf13b731b01bbd5f ({username='oryx', principal='oryx/127.4.83.124@KRBTEST.COM'} at 127.0.0.1:56564)
I20260619 20:32:09.587325 10558 tablet_copy_service.cc:161] P c8a05ff823034550b48979e189dfdf11: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 3ee0d6ea97c54966bf13b731b01bbd5f at {username='oryx', principal='oryx/127.4.83.124@KRBTEST.COM'} at 127.0.0.1:56564: session id = 3ee0d6ea97c54966bf13b731b01bbd5f-00000000000000000000000000000000
I20260619 20:32:09.587785 10558 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 20:32:09.588277 11190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 20:32:09.589228 11190 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f: tablet copy: Starting download of 0 data blocks...
I20260619 20:32:09.589366 11190 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f: tablet copy: Starting download of 1 WAL segments...
I20260619 20:32:09.590027 11190 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 20:32:09.590848 10558 tablet_copy_service.cc:342] P c8a05ff823034550b48979e189dfdf11: Request end of tablet copy session 3ee0d6ea97c54966bf13b731b01bbd5f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.4.83.124@KRBTEST.COM'} at 127.0.0.1:56564
I20260619 20:32:09.590909 10558 tablet_copy_service.cc:433] P c8a05ff823034550b48979e189dfdf11: ending tablet copy session 3ee0d6ea97c54966bf13b731b01bbd5f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 3ee0d6ea97c54966bf13b731b01bbd5f
I20260619 20:32:09.593240 11190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:09.594575 11295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:09.602653 11296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:09.602713 11298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:09.602933 11190 server_base.cc:1034] running on GCE node
I20260619 20:32:09.603076 11190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:09.603235 11190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:09.604370 11190 hybrid_clock.cc:648] HybridClock initialized: now 1781901129604360 us; error 26 us; skew 500 ppm
Jun 19 20:32:09 dist-test-slave-4m58 krb5kdc[10493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901129, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:09.605753 11190 init.cc:377] Logged in from keytab as oryx/127.4.83.124@KRBTEST.COM (short username oryx)
I20260619 20:32:09.606053 11190 webserver.cc:460] Webserver started at http://127.4.83.124:40267/ using document root <none> and password file <none>
I20260619 20:32:09.606169 11190 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:09.606202 11190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:09.606786 11190 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:09.607278 11305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:09.607402 11190 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:09.607447 11190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-2/wal
uuid: "3ee0d6ea97c54966bf13b731b01bbd5f"
format_stamp: "Formatted at 2026-06-19 20:32:09 on dist-test-slave-4m58"
server_key: "8eef77044cad67691795cbc57edc0d5e"
server_key_iv: "bdc8f906d54b8daa6a0a3035e648e385"
server_key_version: "encryptionkey@0"
I20260619 20:32:09.607509 11190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:09.629546 11190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:09.631152 11190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:09.633955 11190 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.124:45667
I20260619 20:32:09.634372 11190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/master-2/data/info.pb
I20260619 20:32:09.634682 11357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.124:45667 every 8 connection(s)
I20260619 20:32:09.635322 11358 sys_catalog.cc:263] Verifying existing consensus state
I20260619 20:32:09.635716 11358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f: Bootstrap starting.
I20260619 20:32:09.638286 11358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:32:09.638511 11358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f: Bootstrap complete.
I20260619 20:32:09.638715 11358 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: false } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: NON_VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: true } }
I20260619 20:32:09.638792 11358 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3ee0d6ea97c54966bf13b731b01bbd5f, State: Initialized, Role: LEARNER
I20260619 20:32:09.638829 11358 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: false } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: NON_VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: true } }
I20260619 20:32:09.639051 11358 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:32:09.639022 11361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: false } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: NON_VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: true } } }
I20260619 20:32:09.639214 11361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [sys.catalog]: This master's current role is: LEARNER
I20260619 20:32:09.640687 11372 catalog_manager.cc:1259] Loaded cluster ID: 32dbae523902416f99c2f5cbddadd76c
I20260619 20:32:09.640746 11372 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f: loading cluster ID for follower catalog manager: success
I20260619 20:32:09.641806 11372 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f: acquiring CA information for follower catalog manager: success
I20260619 20:32:09.642109 11372 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 20:32:09.670650 11283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:09.670935 11283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:09.670969 11283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:09.672797 11283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:09.672859 11283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:09.672902 11283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:32:09.675231 11283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:44621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.65.keytab
--principal=oryx/127.4.83.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:35511
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=38353
--tserver_master_addrs=127.4.83.126:35083,127.4.83.125:44301,127.4.83.124:45667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:09.675520 11283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:09.675797 11283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:09.681645 11378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:09.681811 11377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:09.681783 11283 server_base.cc:1034] running on GCE node
W20260619 20:32:09.681663 11380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:09.682130 11283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:09.682427 11283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:09.683580 11283 hybrid_clock.cc:648] HybridClock initialized: now 1781901129683591 us; error 54 us; skew 500 ppm
Jun 19 20:32:09 dist-test-slave-4m58 krb5kdc[10493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901129, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:09.686926 11283 init.cc:377] Logged in from keytab as oryx/127.4.83.65@KRBTEST.COM (short username oryx)
I20260619 20:32:09.687531 11283 webserver.cc:460] Webserver started at http://127.4.83.65:38353/ using document root <none> and password file <none>
I20260619 20:32:09.687793 11283 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:09.687894 11283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:09.689606 11283 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:32:09.690274 11387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:09.690445 11283 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:32:09.690539 11283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "50ef078492ce4fa6a41e0f986ce8390d"
format_stamp: "Formatted at 2026-06-19 20:32:08 on dist-test-slave-4m58"
server_key: "1b977f914dd9e6c468b3920b681b5e83"
server_key_iv: "b15dd9b8a6a728e4829e7e568d7d34e5"
server_key_version: "encryptionkey@0"
I20260619 20:32:09.690984 11283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:09.701359 11283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:09.703308 11283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:09.703425 11283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:09.703657 11283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:09.704021 11283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:09.704056 11283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:09.704085 11283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:09.704097 11283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:09.709584 11283 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:35511
I20260619 20:32:09.709643 11500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:35511 every 8 connection(s)
I20260619 20:32:09.709961 11283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
I20260619 20:32:09.715680 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 11283
I20260619 20:32:09.715865 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 10734
Jun 19 20:32:09 dist-test-slave-4m58 krb5kdc[10493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901129, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.125@KRBTEST.COM
Jun 19 20:32:09 dist-test-slave-4m58 krb5kdc[10493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901129, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.124@KRBTEST.COM
Jun 19 20:32:09 dist-test-slave-4m58 krb5kdc[10493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901129, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM
I20260619 20:32:09.721307 11187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.711063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:59371 (local address 127.4.83.125:44301)
0619 20:32:09.711130 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.711132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.711140 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:32:09.712022 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.712023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.712036 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.712120 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:09.712890 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.713672 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.714612 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.714902 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.714966 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.717751 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:09.717753 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:09.717754 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:09.717763 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:09.719349 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.719901 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.719903 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.719904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.719972 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.720434 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.720435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.720435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.720613 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:09.720667 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.721101 (+ 434us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.721207 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 20:32:09.721967 11507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.710681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:44301 (local address 127.4.83.65:59371)
0619 20:32:09.710868 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.710894 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:09.711894 (+ 1000us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:09.712194 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:09.712197 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:09.712548 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:09.712790 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.712795 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.713760 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.713761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.714526 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.714529 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.714977 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.714977 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.715145 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.715820 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:09.716075 (+ 255us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:09.717622 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:09.719439 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.719449 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.719452 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.719772 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.720148 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.720149 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.720150 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.720305 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.720703 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:09.720710 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:09.720892 (+ 182us) client_negotiation.cc:769] Sending connection context
0619 20:32:09.721525 (+ 633us) client_negotiation.cc:240] Negotiation successful
0619 20:32:09.721726 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":37,"thread_start_us":95,"threads_started":1}
I20260619 20:32:09.721987 10588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.711022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:46977 (local address 127.4.83.126:35083)
0619 20:32:09.711068 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.711070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.711207 (+ 137us) server_negotiation.cc:407] Connection header received
0619 20:32:09.712031 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.712032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.712043 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.712072 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:09.712880 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.713401 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.714651 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.714945 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.714997 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.718897 (+ 3900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:09.718899 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:09.718900 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:09.718908 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:09.720693 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.721189 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.721190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.721191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.721255 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.721436 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.721437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.721437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.721567 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:09.721623 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.721817 (+ 194us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.721896 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:32:09.722287 11508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.710950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.4.83.65:46977)
0619 20:32:09.711147 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.711156 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:09.711921 (+ 765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:09.712193 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:09.712197 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:09.712545 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:09.712789 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.712793 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.713505 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.713505 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.714532 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.714534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.715048 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.715049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.715146 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.715901 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:09.715909 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:09.718773 (+ 2864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:09.720783 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.720784 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.720784 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.721105 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.721320 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.721320 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.721321 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.721376 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.721669 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:09.721669 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:09.721714 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 20:32:09.722083 (+ 369us) client_negotiation.cc:240] Negotiation successful
0619 20:32:09.722188 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":14,"thread_start_us":53,"threads_started":1}
I20260619 20:32:09.722496 11503 heartbeater.cc:347] Connected to a master server at 127.4.83.125:44301
I20260619 20:32:09.722601 11503 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:09.722658 11502 heartbeater.cc:347] Connected to a master server at 127.4.83.126:35083
I20260619 20:32:09.722709 11502 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:09.722801 11503 heartbeater.cc:511] Master 127.4.83.125:44301 requested a full tablet report, sending...
I20260619 20:32:09.722815 11502 heartbeater.cc:511] Master 127.4.83.126:35083 requested a full tablet report, sending...
I20260619 20:32:09.723044 11505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.710691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:39341 (local address 127.4.83.124:45667)
0619 20:32:09.710813 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.710815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.711054 (+ 239us) server_negotiation.cc:407] Connection header received
0619 20:32:09.712076 (+ 1022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.712077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.712092 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.712151 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:09.712885 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.713874 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.714799 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.715019 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.715087 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.718758 (+ 3671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:09.718760 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:09.718761 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:09.718770 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:09.720688 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.721146 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.721147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.721147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.721208 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.721654 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.721655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.721656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.721792 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:09.721850 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.722865 (+ 1015us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.722954 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":55,"threads_started":1}
I20260619 20:32:09.723381 11136 ts_manager.cc:194] Registered new tserver with Master: 50ef078492ce4fa6a41e0f986ce8390d (127.4.83.65:35511)
I20260619 20:32:09.723423 11506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.710662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:45667 (local address 127.4.83.65:39341)
0619 20:32:09.710844 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.710894 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:09.711894 (+ 1000us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:09.712197 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:09.712198 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:09.712548 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:09.712790 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.712795 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.713948 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.713949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.714733 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.714735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.715089 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.715090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.715148 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.715988 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:09.715994 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:09.718466 (+ 2472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:09.720761 (+ 2295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.720762 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.720762 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.721066 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.721271 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.721271 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.721272 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.721324 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.721875 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:09.721875 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:09.721913 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 20:32:09.721969 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 20:32:09.722795 (+ 826us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":4,"thread_start_us":98,"threads_started":1}
I20260619 20:32:09.723640 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:33863
--local_ip_for_outbound_sockets=127.4.83.66
--tserver_master_addrs=127.4.83.126:35083
--webserver_port=43981
--webserver_interface=127.4.83.66
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.66.keytab
--principal=oryx/127.4.83.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:44621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.83.126:35083,127.4.83.125:44301
--tserver_master_addrs=127.4.83.126:35083,127.4.83.125:44301,127.4.83.124:45667 with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:32:09.723681 10537 ts_manager.cc:194] Re-registered known tserver with Master: 50ef078492ce4fa6a41e0f986ce8390d (127.4.83.65:35511)
I20260619 20:32:09.723783 11501 heartbeater.cc:347] Connected to a master server at 127.4.83.124:45667
I20260619 20:32:09.723855 11501 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:09.723923 11501 heartbeater.cc:511] Master 127.4.83.124:45667 requested a full tablet report, sending...
I20260619 20:32:09.724148 10537 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.83.65@KRBTEST.COM'} at 127.4.83.65:46977
I20260619 20:32:09.724259 11322 ts_manager.cc:194] Registered new tserver with Master: 50ef078492ce4fa6a41e0f986ce8390d (127.4.83.65:35511)
W20260619 20:32:09.804755 11509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:09.804979 11509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:09.805013 11509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:09.806282 11509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:09.806334 11509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:09.806371 11509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:32:09.808096 11509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:44621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.66.keytab
--principal=oryx/127.4.83.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:33863
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=43981
--tserver_master_addrs=127.4.83.126:35083,127.4.83.125:44301,127.4.83.124:45667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:09.808404 11509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:09.808676 11509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:09.814210 11515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:09.814255 11509 server_base.cc:1034] running on GCE node
W20260619 20:32:09.814141 11514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:09.814139 11517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:09.814639 11509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:09.814893 11509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:09.816037 11509 hybrid_clock.cc:648] HybridClock initialized: now 1781901129816013 us; error 38 us; skew 500 ppm
Jun 19 20:32:09 dist-test-slave-4m58 krb5kdc[10493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901129, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:09.818709 11509 init.cc:377] Logged in from keytab as oryx/127.4.83.66@KRBTEST.COM (short username oryx)
I20260619 20:32:09.819190 11509 webserver.cc:460] Webserver started at http://127.4.83.66:43981/ using document root <none> and password file <none>
I20260619 20:32:09.819420 11509 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:09.819489 11509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:09.820905 11509 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:09.821494 11524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:09.821741 11509 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:09.821823 11509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b"
format_stamp: "Formatted at 2026-06-19 20:32:08 on dist-test-slave-4m58"
server_key: "11b36dbf68f6c1d2a67c4a29b8cddd82"
server_key_iv: "62d1a737d8fa26b27927e41b61300bbd"
server_key_version: "encryptionkey@0"
I20260619 20:32:09.822175 11509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:09.830667 11509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:09.832683 11509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:09.832814 11509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:09.833016 11509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:09.833302 11509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:09.833334 11509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:09.833364 11509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:09.833387 11509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:09.838919 11509 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:33863
I20260619 20:32:09.838990 11637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:33863 every 8 connection(s)
I20260619 20:32:09.839309 11509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
Jun 19 20:32:09 dist-test-slave-4m58 krb5kdc[10493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901129, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.125@KRBTEST.COM
Jun 19 20:32:09 dist-test-slave-4m58 krb5kdc[10493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901129, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.124@KRBTEST.COM
Jun 19 20:32:09 dist-test-slave-4m58 krb5kdc[10493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901129, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM
I20260619 20:32:09.848564 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 11509
I20260619 20:32:09.848728 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 10868
I20260619 20:32:09.849982 11187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.840515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:43381 (local address 127.4.83.125:44301)
0619 20:32:09.840559 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.840560 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.840571 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:32:09.841640 (+ 1069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.841642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.841651 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.841692 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:09.842297 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.843064 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.844103 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.844293 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.844347 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.846786 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:09.846788 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:09.846788 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:09.846796 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:09.848432 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.848873 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.848875 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.848876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.848937 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.849232 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.849232 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.849233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.849390 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:09.849428 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.849810 (+ 382us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.849890 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:32:09.850514 11643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.840088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:44301 (local address 127.4.83.66:43381)
0619 20:32:09.840245 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.840280 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:09.841383 (+ 1103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:09.841715 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:09.841716 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:09.841985 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:09.842206 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.842211 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.843143 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.843143 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.844046 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.844047 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.844351 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.844352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.844453 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.845066 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:09.845076 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:09.846687 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:09.848497 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.848505 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.848508 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.848779 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.849018 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.849018 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.849019 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.849155 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.849471 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:09.849477 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:09.849648 (+ 171us) client_negotiation.cc:769] Sending connection context
0619 20:32:09.850081 (+ 433us) client_negotiation.cc:240] Negotiation successful
0619 20:32:09.850284 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":49,"thread_start_us":93,"threads_started":1}
I20260619 20:32:09.850673 11505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.840080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:37393 (local address 127.4.83.124:45667)
0619 20:32:09.840131 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.840133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.840368 (+ 235us) server_negotiation.cc:407] Connection header received
0619 20:32:09.841480 (+ 1112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.841482 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.841499 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.841551 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:09.842315 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.842807 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.844043 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.844337 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.844400 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.847653 (+ 3253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:09.847654 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:09.847654 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:09.847661 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:09.849280 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.849822 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.849824 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.849824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.849884 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.850122 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.850123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.850123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.850256 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:09.850298 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.850483 (+ 185us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.850586 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 20:32:09.851070 11642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.840076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:45667 (local address 127.4.83.66:37393)
0619 20:32:09.840238 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.840279 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:09.841362 (+ 1083us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:09.841613 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:09.841617 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:09.841964 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:09.842208 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.842213 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.842900 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.842901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.843965 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.843967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.844449 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.844450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.844522 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.845204 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:09.845208 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:09.847534 (+ 2326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:09.849554 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.849555 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.849555 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.849742 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.849951 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.849952 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.849952 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.850003 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.850342 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:09.850342 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:09.850383 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 20:32:09.850874 (+ 491us) client_negotiation.cc:240] Negotiation successful
0619 20:32:09.850980 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":1,"thread_start_us":88,"threads_started":1}
I20260619 20:32:09.851313 10588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.840648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:37711 (local address 127.4.83.126:35083)
0619 20:32:09.840709 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.840712 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.840721 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:32:09.841750 (+ 1029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.841751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.841762 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.841813 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:09.842323 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.843064 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.843930 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.844184 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.844240 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.847592 (+ 3352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:09.847593 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:09.847594 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:09.847601 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:09.849279 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.850468 (+ 1189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.850469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.850470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.850526 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.850780 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.850781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.850781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.850916 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:09.850958 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.851145 (+ 187us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.851222 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 20:32:09.851707 11644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.840377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.4.83.66:37711)
0619 20:32:09.840539 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.840549 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:09.841374 (+ 825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:09.841831 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:09.841832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:09.842041 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:09.842208 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.842213 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.843174 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.843175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.843865 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.843867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.844307 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.844308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.844453 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.845134 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:09.845139 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:09.847507 (+ 2368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:09.850151 (+ 2644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.850152 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.850152 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.850384 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.850599 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.850599 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.850600 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.850663 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.851004 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:09.851005 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:09.851047 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 20:32:09.851453 (+ 406us) client_negotiation.cc:240] Negotiation successful
0619 20:32:09.851571 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":23,"thread_start_us":70,"threads_started":1}
I20260619 20:32:09.852093 11639 heartbeater.cc:347] Connected to a master server at 127.4.83.126:35083
I20260619 20:32:09.852182 11639 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:09.852327 11639 heartbeater.cc:511] Master 127.4.83.126:35083 requested a full tablet report, sending...
I20260619 20:32:09.853899 10537 ts_manager.cc:194] Re-registered known tserver with Master: ec3fe6dc2f68409699ccbe96a09f7d8b (127.4.83.66:33863)
I20260619 20:32:09.854348 10537 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.83.66@KRBTEST.COM'} at 127.4.83.66:37711
I20260619 20:32:09.854374 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:40087
--local_ip_for_outbound_sockets=127.4.83.67
--tserver_master_addrs=127.4.83.126:35083
--webserver_port=39347
--webserver_interface=127.4.83.67
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.67.keytab
--principal=oryx/127.4.83.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:44621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.83.126:35083,127.4.83.125:44301
--tserver_master_addrs=127.4.83.126:35083,127.4.83.125:44301,127.4.83.124:45667 with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:32:09.854110 11638 heartbeater.cc:347] Connected to a master server at 127.4.83.124:45667
I20260619 20:32:09.854749 11638 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:09.854815 11638 heartbeater.cc:511] Master 127.4.83.124:45667 requested a full tablet report, sending...
I20260619 20:32:09.853801 11640 heartbeater.cc:347] Connected to a master server at 127.4.83.125:44301
I20260619 20:32:09.855131 11640 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:09.855139 11322 ts_manager.cc:194] Registered new tserver with Master: ec3fe6dc2f68409699ccbe96a09f7d8b (127.4.83.66:33863)
I20260619 20:32:09.855229 11640 heartbeater.cc:511] Master 127.4.83.125:44301 requested a full tablet report, sending...
I20260619 20:32:09.855538 11136 ts_manager.cc:194] Registered new tserver with Master: ec3fe6dc2f68409699ccbe96a09f7d8b (127.4.83.66:33863)
W20260619 20:32:09.935534 11645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:09.935763 11645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:09.935787 11645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:09.936995 11645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:09.937045 11645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:09.937072 11645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:32:09.938566 11645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:44621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.67.keytab
--principal=oryx/127.4.83.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:40087
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=39347
--tserver_master_addrs=127.4.83.126:35083,127.4.83.125:44301,127.4.83.124:45667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:09.938799 11645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:09.939025 11645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:09.944147 11651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:09.944262 11645 server_base.cc:1034] running on GCE node
W20260619 20:32:09.944134 11653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:09.944110 11650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:09.944628 11645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:09.944868 11645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:09.946058 11645 hybrid_clock.cc:648] HybridClock initialized: now 1781901129946039 us; error 31 us; skew 500 ppm
Jun 19 20:32:09 dist-test-slave-4m58 krb5kdc[10493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901129, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:09.948864 11645 init.cc:377] Logged in from keytab as oryx/127.4.83.67@KRBTEST.COM (short username oryx)
I20260619 20:32:09.949368 11645 webserver.cc:460] Webserver started at http://127.4.83.67:39347/ using document root <none> and password file <none>
I20260619 20:32:09.949571 11645 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:09.949661 11645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:09.951083 11645 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:09.951785 11660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:09.952057 11645 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:09.952150 11645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "46234eb18fcf465f96ba823f2d6753d3"
format_stamp: "Formatted at 2026-06-19 20:32:08 on dist-test-slave-4m58"
server_key: "8b077d089681b35c1a94e21cc2ca71de"
server_key_iv: "cf064913d829b1e96495bb283285207f"
server_key_version: "encryptionkey@0"
I20260619 20:32:09.952548 11645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:09.963757 11645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:09.965821 11645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:09.965940 11645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:09.966182 11645 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:09.966481 11645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:09.966516 11645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:09.966545 11645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:09.966568 11645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:09.972146 11645 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:40087
I20260619 20:32:09.972198 11773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:40087 every 8 connection(s)
I20260619 20:32:09.972527 11645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
I20260619 20:32:09.979552 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 11645
Jun 19 20:32:09 dist-test-slave-4m58 krb5kdc[10493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901129, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.125@KRBTEST.COM
Jun 19 20:32:09 dist-test-slave-4m58 krb5kdc[10493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901129, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.124@KRBTEST.COM
Jun 19 20:32:09 dist-test-slave-4m58 krb5kdc[10493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901129, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM
I20260619 20:32:09.984947 11187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.973673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:46289 (local address 127.4.83.125:44301)
0619 20:32:09.973716 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.973718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.973728 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:32:09.974616 (+ 888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.974618 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.974632 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.974688 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:09.975571 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.976127 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.977578 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.977777 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.977845 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.981652 (+ 3807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:09.981654 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:09.981654 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:09.981663 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:09.983292 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.983872 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.983874 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.983874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.983931 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.984243 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.984243 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.984244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.984372 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:09.984408 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.984774 (+ 366us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.984855 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:32:09.985477 11780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.973548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:44301 (local address 127.4.83.67:46289)
0619 20:32:09.973687 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.973696 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:09.974494 (+ 798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:09.974772 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:09.974776 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:09.975247 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:09.975456 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.975462 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.976234 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.976235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.977475 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.977478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.977868 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.977869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.978011 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.978653 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:09.978663 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:09.981554 (+ 2891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:09.983419 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.983428 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.983430 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.983762 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.984003 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.984004 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.984004 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.984174 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.984447 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:09.984453 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:09.984636 (+ 183us) client_negotiation.cc:769] Sending connection context
0619 20:32:09.985075 (+ 439us) client_negotiation.cc:240] Negotiation successful
0619 20:32:09.985255 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":16,"thread_start_us":69,"threads_started":1}
I20260619 20:32:09.985955 11776 heartbeater.cc:347] Connected to a master server at 127.4.83.125:44301
I20260619 20:32:09.986056 11776 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:09.986243 11776 heartbeater.cc:511] Master 127.4.83.125:44301 requested a full tablet report, sending...
I20260619 20:32:09.986690 11136 ts_manager.cc:194] Registered new tserver with Master: 46234eb18fcf465f96ba823f2d6753d3 (127.4.83.67:40087)
I20260619 20:32:09.987787 11779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.973251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.4.83.67:44907)
0619 20:32:09.973596 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.973606 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:09.974500 (+ 894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:09.974776 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:09.974777 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:09.975199 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:09.975456 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.975461 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.976427 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.976428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.977475 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.977478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.978187 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.978188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.978263 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.978750 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:09.978755 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:09.981548 (+ 2793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:09.983993 (+ 2445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.983994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.983994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.984200 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.986079 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.986080 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.986081 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.986142 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.987446 (+ 1304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:09.987447 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:09.987490 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 20:32:09.987561 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 20:32:09.987671 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":63,"thread_start_us":95,"threads_started":1}
I20260619 20:32:09.988051 11505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.973248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:37027 (local address 127.4.83.124:45667)
0619 20:32:09.973313 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.973315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.973744 (+ 429us) server_negotiation.cc:407] Connection header received
0619 20:32:09.974740 (+ 996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.974742 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.974753 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.974817 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:09.975572 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.976358 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.978346 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.978595 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.978658 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.981283 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:09.981285 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:09.981285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:09.981294 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:09.986277 (+ 4983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.987093 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.987094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.987094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.987150 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.987337 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.987338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.987338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.987462 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:09.987514 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.987880 (+ 366us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.987968 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 20:32:09.988551 11778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.973251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:45667 (local address 127.4.83.67:37027)
0619 20:32:09.973436 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.973479 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:09.974467 (+ 988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:09.974831 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:09.974831 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:09.975231 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:09.975456 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.975462 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.977574 (+ 2112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.977574 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.978213 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.978215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.978715 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.978716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.978779 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.979148 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:09.979152 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:09.981139 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:09.986386 (+ 5247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.986387 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.986387 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.987006 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.987213 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:09.987214 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:09.987214 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:09.987273 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:09.987536 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:09.987537 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:09.987583 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 20:32:09.987638 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 20:32:09.987713 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":27,"thread_start_us":95,"threads_started":1}
I20260619 20:32:09.988864 10588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.973248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:44907 (local address 127.4.83.126:35083)
0619 20:32:09.973353 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.973355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.973666 (+ 311us) server_negotiation.cc:407] Connection header received
0619 20:32:09.974629 (+ 963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.974631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.974643 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.974673 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:09.975561 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.976346 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.977906 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.978105 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.978167 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.981641 (+ 3474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:09.981642 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:09.981642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:09.981650 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:09.983903 (+ 2253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.985909 (+ 2006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.985910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.985911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.985976 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.987220 (+ 1244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.987221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.987221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.987364 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:09.987422 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.988689 (+ 1267us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.988771 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260619 20:32:09.989197 11774 heartbeater.cc:347] Connected to a master server at 127.4.83.124:45667
I20260619 20:32:09.989216 11787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.980729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49082 (local address 127.4.83.124:45667)
0619 20:32:09.980895 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.980897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.981214 (+ 317us) server_negotiation.cc:407] Connection header received
0619 20:32:09.982049 (+ 835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.982049 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.982062 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.982102 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:09.982802 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.983556 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.984556 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.984868 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.984907 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.986311 (+ 1404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:09.986312 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:09.986312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:09.986319 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:09.987889 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.988285 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.988286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.988286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.988342 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.988498 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.988499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.988500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.988628 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:09.988721 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.988887 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.989124 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":50,"threads_started":1}
I20260619 20:32:09.989257 11774 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:09.989269 11786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.980597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56580 (local address 127.4.83.126:35083)
0619 20:32:09.980847 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.980849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.981011 (+ 162us) server_negotiation.cc:407] Connection header received
0619 20:32:09.981116 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.981117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.981132 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.981177 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:09.982470 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.983293 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.984364 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.984657 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.984700 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.986259 (+ 1559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:09.986260 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:09.986260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:09.986267 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:09.987986 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.988344 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.988345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.988346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.988402 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.988576 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.988577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.988577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.988843 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:09.988923 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.989102 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.989181 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":80,"threads_started":1}
I20260619 20:32:09.989334 11774 heartbeater.cc:511] Master 127.4.83.124:45667 requested a full tablet report, sending...
I20260619 20:32:09.989653 11322 ts_manager.cc:194] Registered new tserver with Master: 46234eb18fcf465f96ba823f2d6753d3 (127.4.83.67:40087)
I20260619 20:32:09.989850 11775 heartbeater.cc:347] Connected to a master server at 127.4.83.126:35083
I20260619 20:32:09.989897 11775 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:09.989959 11775 heartbeater.cc:511] Master 127.4.83.126:35083 requested a full tablet report, sending...
I20260619 20:32:09.990420 10537 ts_manager.cc:194] Re-registered known tserver with Master: 46234eb18fcf465f96ba823f2d6753d3 (127.4.83.67:40087)
I20260619 20:32:09.990945 10537 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.83.67@KRBTEST.COM'} at 127.4.83.67:44907
I20260619 20:32:09.991281 10537 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56580:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:32:09.993078 10537 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:32:09.996189 11789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.981056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52510 (local address 127.4.83.125:44301)
0619 20:32:09.981920 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.981922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.989048 (+ 7126us) server_negotiation.cc:407] Connection header received
0619 20:32:09.989138 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.989139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.989152 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.989189 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:09.989793 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.990621 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.991891 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.992143 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.992204 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:09.993379 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:09.993380 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:09.993381 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:09.993388 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:09.994945 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.995337 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.995338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.995338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.995388 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:09.995561 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:09.995561 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:09.995562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:09.995702 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:09.995826 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:09.995997 (+ 171us) server_negotiation.cc:299] Negotiation successful
0619 20:32:09.996086 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":842,"thread_start_us":51,"threads_started":1}
I20260619 20:32:10.000954 11096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.996885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:40087 (local address 127.0.0.1:34668)
0619 20:32:09.996918 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.996930 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:09.996970 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:09.997591 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:09.997592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:09.997594 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:09.997647 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.997649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.998805 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.998806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:09.999785 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.999788 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:10.000730 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:10.000731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:10.000790 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:10.000794 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:32:10.000826 (+ 32us) client_negotiation.cc:240] Negotiation successful
0619 20:32:10.000869 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260619 20:32:10.000977 11798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.997211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34668 (local address 127.4.83.67:40087)
0619 20:32:09.997394 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.997398 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.997409 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:32:09.997455 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.997458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.997476 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.997743 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:09.997800 (+ 57us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.998733 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.999911 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:10.000637 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:10.000719 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:10.000816 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:10.000854 (+ 38us) server_negotiation.cc:299] Negotiation successful
0619 20:32:10.000892 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":90,"threads_started":1}
I20260619 20:32:10.001577 11799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.997235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48798 (local address 127.4.83.65:35511)
0619 20:32:09.997392 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.997396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.997600 (+ 204us) server_negotiation.cc:407] Connection header received
0619 20:32:09.997646 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.997649 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.997665 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.997735 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:09.997876 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.998966 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:10.000182 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:10.000754 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:10.000815 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:10.000878 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:10.001475 (+ 597us) server_negotiation.cc:299] Negotiation successful
0619 20:32:10.001513 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":81,"threads_started":1}
I20260619 20:32:10.001649 11795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.996884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:33863 (local address 127.0.0.1:37938)
0619 20:32:09.997386 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.997396 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:09.997605 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:09.997843 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:09.997843 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:09.997845 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:09.997899 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.997901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.999151 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.999152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:10.000256 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:10.000259 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:10.001410 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:10.001411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:10.001483 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:10.001488 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:32:10.001536 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 20:32:10.001577 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"mutex_wait_us":22,"thread_start_us":125,"threads_started":1}
I20260619 20:32:10.001782 11797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.997131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:35511 (local address 127.0.0.1:48798)
0619 20:32:09.997496 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:09.997506 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:09.997555 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:09.997746 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:09.997747 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:09.997750 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:09.997797 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:09.997799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.999070 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:09.999071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:10.000099 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:10.000102 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:10.000908 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:10.000909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:10.000962 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:10.000966 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:32:10.001006 (+ 40us) client_negotiation.cc:240] Negotiation successful
0619 20:32:10.001698 (+ 692us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":30,"thread_start_us":71,"threads_started":1}
I20260619 20:32:10.001931 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:09.996987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37938 (local address 127.4.83.66:33863)
0619 20:32:09.997214 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:09.997219 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:09.997453 (+ 234us) server_negotiation.cc:407] Connection header received
0619 20:32:09.997691 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:09.997693 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:09.997710 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:09.997757 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:09.997971 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:09.999064 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:10.000330 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:10.001014 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:10.001075 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:10.001370 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:10.001807 (+ 437us) server_negotiation.cc:299] Negotiation successful
0619 20:32:10.001855 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":116,"threads_started":1}
I20260619 20:32:10.001863 11708 tablet_service.cc:1467] Processing CreateTablet for tablet 44ddf6de46cf46a3a9b2d8bfb054f6d7 (DEFAULT_TABLE table=test-table [id=ad78355097984a86a5ebded1aba328de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:32:10.002262 11708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44ddf6de46cf46a3a9b2d8bfb054f6d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:10.002589 11572 tablet_service.cc:1467] Processing CreateTablet for tablet 44ddf6de46cf46a3a9b2d8bfb054f6d7 (DEFAULT_TABLE table=test-table [id=ad78355097984a86a5ebded1aba328de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:32:10.002840 11572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44ddf6de46cf46a3a9b2d8bfb054f6d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:10.004544 11435 tablet_service.cc:1467] Processing CreateTablet for tablet 44ddf6de46cf46a3a9b2d8bfb054f6d7 (DEFAULT_TABLE table=test-table [id=ad78355097984a86a5ebded1aba328de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:32:10.004874 11435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44ddf6de46cf46a3a9b2d8bfb054f6d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:10.005159 11800 tablet_bootstrap.cc:492] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3: Bootstrap starting.
I20260619 20:32:10.005824 11800 tablet_bootstrap.cc:654] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:10.005954 11801 tablet_bootstrap.cc:492] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P ec3fe6dc2f68409699ccbe96a09f7d8b: Bootstrap starting.
I20260619 20:32:10.006124 11800 log.cc:824] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:10.006533 11801 tablet_bootstrap.cc:654] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P ec3fe6dc2f68409699ccbe96a09f7d8b: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:10.006808 11800 tablet_bootstrap.cc:492] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3: No bootstrap required, opened a new log
I20260619 20:32:10.006909 11800 ts_tablet_manager.cc:1397] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:32:10.006983 11801 log.cc:824] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P ec3fe6dc2f68409699ccbe96a09f7d8b: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:10.008044 11801 tablet_bootstrap.cc:492] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P ec3fe6dc2f68409699ccbe96a09f7d8b: No bootstrap required, opened a new log
I20260619 20:32:10.008109 11801 ts_tablet_manager.cc:1397] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P ec3fe6dc2f68409699ccbe96a09f7d8b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:32:10.008548 11804 tablet_bootstrap.cc:492] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 50ef078492ce4fa6a41e0f986ce8390d: Bootstrap starting.
I20260619 20:32:10.008837 11800 raft_consensus.cc:348] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33863 } } peers { permanent_uuid: "46234eb18fcf465f96ba823f2d6753d3" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40087 } } peers { permanent_uuid: "50ef078492ce4fa6a41e0f986ce8390d" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 35511 } }
I20260619 20:32:10.008975 11800 raft_consensus.cc:374] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:10.009003 11800 raft_consensus.cc:729] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46234eb18fcf465f96ba823f2d6753d3, State: Initialized, Role: FOLLOWER
I20260619 20:32:10.009138 11800 consensus_queue.cc:260] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33863 } } peers { permanent_uuid: "46234eb18fcf465f96ba823f2d6753d3" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40087 } } peers { permanent_uuid: "50ef078492ce4fa6a41e0f986ce8390d" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 35511 } }
I20260619 20:32:10.009181 11804 tablet_bootstrap.cc:654] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 50ef078492ce4fa6a41e0f986ce8390d: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:10.009387 11800 ts_tablet_manager.cc:1428] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:32:10.009449 11804 log.cc:824] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 50ef078492ce4fa6a41e0f986ce8390d: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:10.009507 11775 heartbeater.cc:503] Master 127.4.83.126:35083 was elected leader, sending a full tablet report...
I20260619 20:32:10.010150 11804 tablet_bootstrap.cc:492] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 50ef078492ce4fa6a41e0f986ce8390d: No bootstrap required, opened a new log
I20260619 20:32:10.010207 11804 ts_tablet_manager.cc:1397] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 50ef078492ce4fa6a41e0f986ce8390d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:32:10.010310 11801 raft_consensus.cc:348] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P ec3fe6dc2f68409699ccbe96a09f7d8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33863 } } peers { permanent_uuid: "46234eb18fcf465f96ba823f2d6753d3" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40087 } } peers { permanent_uuid: "50ef078492ce4fa6a41e0f986ce8390d" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 35511 } }
I20260619 20:32:10.010460 11801 raft_consensus.cc:374] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P ec3fe6dc2f68409699ccbe96a09f7d8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:10.010489 11801 raft_consensus.cc:729] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P ec3fe6dc2f68409699ccbe96a09f7d8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec3fe6dc2f68409699ccbe96a09f7d8b, State: Initialized, Role: FOLLOWER
I20260619 20:32:10.010607 11801 consensus_queue.cc:260] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P ec3fe6dc2f68409699ccbe96a09f7d8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33863 } } peers { permanent_uuid: "46234eb18fcf465f96ba823f2d6753d3" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40087 } } peers { permanent_uuid: "50ef078492ce4fa6a41e0f986ce8390d" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 35511 } }
I20260619 20:32:10.010839 11801 ts_tablet_manager.cc:1428] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P ec3fe6dc2f68409699ccbe96a09f7d8b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:32:10.010949 11639 heartbeater.cc:503] Master 127.4.83.126:35083 was elected leader, sending a full tablet report...
I20260619 20:32:10.012115 11804 raft_consensus.cc:348] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 50ef078492ce4fa6a41e0f986ce8390d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33863 } } peers { permanent_uuid: "46234eb18fcf465f96ba823f2d6753d3" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40087 } } peers { permanent_uuid: "50ef078492ce4fa6a41e0f986ce8390d" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 35511 } }
I20260619 20:32:10.012280 11804 raft_consensus.cc:374] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 50ef078492ce4fa6a41e0f986ce8390d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:10.012320 11804 raft_consensus.cc:729] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 50ef078492ce4fa6a41e0f986ce8390d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50ef078492ce4fa6a41e0f986ce8390d, State: Initialized, Role: FOLLOWER
I20260619 20:32:10.012429 11804 consensus_queue.cc:260] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 50ef078492ce4fa6a41e0f986ce8390d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33863 } } peers { permanent_uuid: "46234eb18fcf465f96ba823f2d6753d3" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40087 } } peers { permanent_uuid: "50ef078492ce4fa6a41e0f986ce8390d" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 35511 } }
I20260619 20:32:10.012678 11804 ts_tablet_manager.cc:1428] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 50ef078492ce4fa6a41e0f986ce8390d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 20:32:10.012737 11502 heartbeater.cc:503] Master 127.4.83.126:35083 was elected leader, sending a full tablet report...
W20260619 20:32:10.090371 11641 tablet.cc:2369] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P ec3fe6dc2f68409699ccbe96a09f7d8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:32:10.132640 11795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:10.130017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:45667 (local address 127.0.0.1:49096)
0619 20:32:10.130092 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:10.130103 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:10.130160 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:10.130295 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:10.130296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:10.130297 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:10.130347 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:10.130349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:10.130949 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:10.130949 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:10.131691 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:10.131694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:10.132392 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:10.132394 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:10.132460 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:10.132465 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:32:10.132522 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 20:32:10.132572 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 20:32:10.132702 11787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:10.130091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49096 (local address 127.4.83.124:45667)
0619 20:32:10.130151 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:10.130153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:10.130163 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:32:10.130207 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:10.130208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:10.130227 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:10.130285 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:10.130395 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:10.130893 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:10.131812 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:10.132282 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:10.132363 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:10.132434 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:10.132564 (+ 130us) server_negotiation.cc:299] Negotiation successful
0619 20:32:10.132640 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 20:32:10.133136 11332 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260619 20:32:10.134006 11361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8a05ff823034550b48979e189dfdf11. Latest consensus state: current_term: 1 leader_uuid: "c8a05ff823034550b48979e189dfdf11" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: false } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: NON_VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: true } } }
I20260619 20:32:10.134114 11361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [sys.catalog]: This master's current role is: LEARNER
W20260619 20:32:10.210805 11504 tablet.cc:2369] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 50ef078492ce4fa6a41e0f986ce8390d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:32:10.223994 11777 tablet.cc:2369] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:32:10.260160 11805 raft_consensus.cc:482] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:32:10.260316 11805 raft_consensus.cc:504] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33863 } } peers { permanent_uuid: "46234eb18fcf465f96ba823f2d6753d3" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40087 } } peers { permanent_uuid: "50ef078492ce4fa6a41e0f986ce8390d" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 35511 } }
I20260619 20:32:10.260681 11805 leader_election.cc:288] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec3fe6dc2f68409699ccbe96a09f7d8b (127.4.83.66:33863), 50ef078492ce4fa6a41e0f986ce8390d (127.4.83.65:35511)
I20260619 20:32:10.263616 11799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:10.261147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:54187 (local address 127.4.83.65:35511)
0619 20:32:10.261191 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:10.261192 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:10.261203 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:32:10.261236 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:10.261237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:10.261245 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:10.261282 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:10.261435 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:10.261910 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:10.262954 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:10.263313 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:10.263376 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:10.263385 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:10.263515 (+ 130us) server_negotiation.cc:299] Negotiation successful
0619 20:32:10.263549 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 20:32:10.263653 11778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:10.260904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:33863 (local address 127.4.83.67:48213)
0619 20:32:10.260975 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:10.260989 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:10.261047 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:10.261315 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:10.261316 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:10.261318 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:10.261377 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:10.261378 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:10.261991 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:10.261992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:10.262804 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:10.262806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:10.263377 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:10.263378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:10.263445 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:10.263448 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:32:10.263491 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 20:32:10.263543 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":38}
I20260619 20:32:10.263660 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:10.261131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:48213 (local address 127.4.83.66:33863)
0619 20:32:10.261196 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:10.261197 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:10.261207 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:32:10.261234 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:10.261235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:10.261247 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:10.261298 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:10.261442 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:10.261910 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:10.262923 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:10.263305 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:10.263361 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:10.263373 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:10.263511 (+ 138us) server_negotiation.cc:299] Negotiation successful
0619 20:32:10.263595 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 20:32:10.263667 11779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:10.261068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:35511 (local address 127.4.83.67:54187)
0619 20:32:10.261129 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:10.261138 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:10.261183 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:10.261295 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:10.261296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:10.261298 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:10.261369 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:10.261371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:10.261989 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:10.261990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:10.262806 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:10.262809 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:10.263383 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:10.263384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:10.263452 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:10.263455 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:32:10.263502 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 20:32:10.263544 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":29}
I20260619 20:32:10.263985 11455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44ddf6de46cf46a3a9b2d8bfb054f6d7" candidate_uuid: "46234eb18fcf465f96ba823f2d6753d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ef078492ce4fa6a41e0f986ce8390d" is_pre_election: true
I20260619 20:32:10.264022 11592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44ddf6de46cf46a3a9b2d8bfb054f6d7" candidate_uuid: "46234eb18fcf465f96ba823f2d6753d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b" is_pre_election: true
I20260619 20:32:10.264148 11592 raft_consensus.cc:2454] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P ec3fe6dc2f68409699ccbe96a09f7d8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46234eb18fcf465f96ba823f2d6753d3 in term 0.
I20260619 20:32:10.264148 11455 raft_consensus.cc:2454] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 50ef078492ce4fa6a41e0f986ce8390d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46234eb18fcf465f96ba823f2d6753d3 in term 0.
I20260619 20:32:10.264364 11661 leader_election.cc:302] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46234eb18fcf465f96ba823f2d6753d3, 50ef078492ce4fa6a41e0f986ce8390d; no voters:
I20260619 20:32:10.264513 11805 raft_consensus.cc:2781] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:32:10.264616 11805 raft_consensus.cc:482] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:32:10.264635 11805 raft_consensus.cc:3037] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:10.265314 11805 raft_consensus.cc:504] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33863 } } peers { permanent_uuid: "46234eb18fcf465f96ba823f2d6753d3" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40087 } } peers { permanent_uuid: "50ef078492ce4fa6a41e0f986ce8390d" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 35511 } }
I20260619 20:32:10.265427 11805 leader_election.cc:288] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3 [CANDIDATE]: Term 1 election: Requested vote from peers ec3fe6dc2f68409699ccbe96a09f7d8b (127.4.83.66:33863), 50ef078492ce4fa6a41e0f986ce8390d (127.4.83.65:35511)
I20260619 20:32:10.265651 11592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44ddf6de46cf46a3a9b2d8bfb054f6d7" candidate_uuid: "46234eb18fcf465f96ba823f2d6753d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b"
I20260619 20:32:10.265745 11592 raft_consensus.cc:3037] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P ec3fe6dc2f68409699ccbe96a09f7d8b [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:10.266428 11592 raft_consensus.cc:2454] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P ec3fe6dc2f68409699ccbe96a09f7d8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46234eb18fcf465f96ba823f2d6753d3 in term 1.
I20260619 20:32:10.266533 11455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44ddf6de46cf46a3a9b2d8bfb054f6d7" candidate_uuid: "46234eb18fcf465f96ba823f2d6753d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ef078492ce4fa6a41e0f986ce8390d"
I20260619 20:32:10.266608 11455 raft_consensus.cc:3037] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 50ef078492ce4fa6a41e0f986ce8390d [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:10.266625 11661 leader_election.cc:302] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46234eb18fcf465f96ba823f2d6753d3, ec3fe6dc2f68409699ccbe96a09f7d8b; no voters:
I20260619 20:32:10.266741 11805 raft_consensus.cc:2781] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:32:10.266872 11805 raft_consensus.cc:686] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3 [term 1 LEADER]: Becoming Leader. State: Replica: 46234eb18fcf465f96ba823f2d6753d3, State: Running, Role: LEADER
I20260619 20:32:10.267027 11805 consensus_queue.cc:237] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33863 } } peers { permanent_uuid: "46234eb18fcf465f96ba823f2d6753d3" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40087 } } peers { permanent_uuid: "50ef078492ce4fa6a41e0f986ce8390d" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 35511 } }
I20260619 20:32:10.267383 11455 raft_consensus.cc:2454] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 50ef078492ce4fa6a41e0f986ce8390d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46234eb18fcf465f96ba823f2d6753d3 in term 1.
I20260619 20:32:10.267904 10537 catalog_manager.cc:5495] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46234eb18fcf465f96ba823f2d6753d3 (127.4.83.67). New cstate: current_term: 1 leader_uuid: "46234eb18fcf465f96ba823f2d6753d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46234eb18fcf465f96ba823f2d6753d3" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50ef078492ce4fa6a41e0f986ce8390d" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 35511 } health_report { overall_health: UNKNOWN } } }
I20260619 20:32:10.268466 11097 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c8a05ff823034550b48979e189dfdf11" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: false } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: NON_VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: true } } }
I20260619 20:32:10.268563 11097 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:10.269083 11097 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11: attempting to promote NON_VOTER 3ee0d6ea97c54966bf13b731b01bbd5f to VOTER
I20260619 20:32:10.269189 11097 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: false } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: false } }
I20260619 20:32:10.269595 11146 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [term 1 FOLLOWER]: Refusing update from remote peer c8a05ff823034550b48979e189dfdf11: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260619 20:32:10.269640 11331 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [term 1 LEARNER]: Refusing update from remote peer c8a05ff823034550b48979e189dfdf11: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260619 20:32:10.269791 11098 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260619 20:32:10.269878 11284 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260619 20:32:10.270434 11097 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 3ee0d6ea97c54966bf13b731b01bbd5f (127.4.83.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: false } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: false } } }
I20260619 20:32:10.270583 11146 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 3ee0d6ea97c54966bf13b731b01bbd5f (127.4.83.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: false } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: false } } }
I20260619 20:32:10.270639 11331 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 3ee0d6ea97c54966bf13b731b01bbd5f (127.4.83.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: false } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: false } } }
I20260619 20:32:10.271150 11097 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c8a05ff823034550b48979e189dfdf11" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: false } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: false } } }
I20260619 20:32:10.271216 11097 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:10.271270 11097 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c8a05ff823034550b48979e189dfdf11" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: false } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: false } } }
I20260619 20:32:10.271310 11097 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:10.271344 11097 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c8a05ff823034550b48979e189dfdf11" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: false } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: false } } }
I20260619 20:32:10.271382 11097 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:10.271482 11815 sys_catalog.cc:455] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c8a05ff823034550b48979e189dfdf11" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: false } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: false } } }
I20260619 20:32:10.271612 11815 sys_catalog.cc:458] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:32:10.271656 11361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c8a05ff823034550b48979e189dfdf11" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: false } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: false } } }
I20260619 20:32:10.271736 11361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:32:10.288275 11798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:10.285979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34684 (local address 127.4.83.67:40087)
0619 20:32:10.286039 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:10.286041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:10.286051 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:32:10.286112 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:10.286113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:10.286137 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:10.286203 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:32:10.286315 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:10.286806 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:10.287507 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:10.287717 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:10.287774 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:10.287946 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:32:10.288061 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:32:10.288134 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:10.288171 (+ 37us) server_negotiation.cc:299] Negotiation successful
0619 20:32:10.288192 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 20:32:10.290174 11592 raft_consensus.cc:1261] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P ec3fe6dc2f68409699ccbe96a09f7d8b [term 1 FOLLOWER]: Refusing update from remote peer 46234eb18fcf465f96ba823f2d6753d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:32:10.290215 11455 raft_consensus.cc:1261] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 50ef078492ce4fa6a41e0f986ce8390d [term 1 FOLLOWER]: Refusing update from remote peer 46234eb18fcf465f96ba823f2d6753d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:32:10.290530 11812 consensus_queue.cc:1035] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:32:10.290697 11805 consensus_queue.cc:1035] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50ef078492ce4fa6a41e0f986ce8390d" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 35511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:32:10.293526 11817 mvcc.cc:205] Tried to move back new op lower bound from 7298667029665570816 to 7298667029574336512. Current Snapshot: MvccSnapshot[applied={T|T < 7298667029665570816}]
I20260619 20:32:10.293715 11818 mvcc.cc:205] Tried to move back new op lower bound from 7298667029665570816 to 7298667029574336512. Current Snapshot: MvccSnapshot[applied={T|T < 7298667029665570816}]
I20260619 20:32:10.295749 11819 mvcc.cc:205] Tried to move back new op lower bound from 7298667029665570816 to 7298667029574336512. Current Snapshot: MvccSnapshot[applied={T|T < 7298667029665570816 or (T in {7298667029665570816})}]
I20260619 20:32:10.295818 10537 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:32:10.295948 10537 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:32:10.297253 10537 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c8a05ff823034550b48979e189dfdf11: Sending DeleteTablet for 3 replicas of tablet 44ddf6de46cf46a3a9b2d8bfb054f6d7
I20260619 20:32:10.297605 11435 tablet_service.cc:1514] Processing DeleteTablet for tablet 44ddf6de46cf46a3a9b2d8bfb054f6d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:32:10 UTC) from {username='oryx', principal='oryx/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:48798
I20260619 20:32:10.297647 11572 tablet_service.cc:1514] Processing DeleteTablet for tablet 44ddf6de46cf46a3a9b2d8bfb054f6d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:32:10 UTC) from {username='oryx', principal='oryx/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:37938
I20260619 20:32:10.297802 11827 tablet_replica.cc:330] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 50ef078492ce4fa6a41e0f986ce8390d: stopping tablet replica
I20260619 20:32:10.297907 11827 raft_consensus.cc:2229] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 50ef078492ce4fa6a41e0f986ce8390d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:32:10.298061 11827 raft_consensus.cc:2258] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 50ef078492ce4fa6a41e0f986ce8390d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:32:10.298089 11708 tablet_service.cc:1514] Processing DeleteTablet for tablet 44ddf6de46cf46a3a9b2d8bfb054f6d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:32:10 UTC) from {username='oryx', principal='oryx/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:34668
I20260619 20:32:10.298318 11829 tablet_replica.cc:330] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3: stopping tablet replica
I20260619 20:32:10.298446 11829 raft_consensus.cc:2229] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:32:10.298588 11829 raft_consensus.cc:2258] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:32:10.298619 11827 ts_tablet_manager.cc:1905] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 50ef078492ce4fa6a41e0f986ce8390d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:32:10.299240 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 10506
I20260619 20:32:10.300313 11829 ts_tablet_manager.cc:1905] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:32:10.300364 11827 ts_tablet_manager.cc:1918] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 50ef078492ce4fa6a41e0f986ce8390d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:32:10.300434 11827 log.cc:1196] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 50ef078492ce4fa6a41e0f986ce8390d: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-0/wal/wals/44ddf6de46cf46a3a9b2d8bfb054f6d7
I20260619 20:32:10.300900 11827 ts_tablet_manager.cc:1939] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 50ef078492ce4fa6a41e0f986ce8390d: Deleting consensus metadata
I20260619 20:32:10.302510 11829 ts_tablet_manager.cc:1918] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:32:10.302596 11829 log.cc:1196] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/wal/wals/44ddf6de46cf46a3a9b2d8bfb054f6d7
I20260619 20:32:10.302863 11829 ts_tablet_manager.cc:1939] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P 46234eb18fcf465f96ba823f2d6753d3: Deleting consensus metadata
I20260619 20:32:10.305307 11828 tablet_replica.cc:330] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P ec3fe6dc2f68409699ccbe96a09f7d8b: stopping tablet replica
W20260619 20:32:10.305418 11388 connection.cc:535] server connection from 127.0.0.1:48798 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 20:32:10.305462 11828 raft_consensus.cc:2229] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P ec3fe6dc2f68409699ccbe96a09f7d8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:32:10.305589 11828 raft_consensus.cc:2258] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P ec3fe6dc2f68409699ccbe96a09f7d8b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 20:32:10.305688 11664 connection.cc:535] server connection from 127.0.0.1:34668 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 20:32:10.307237 11828 ts_tablet_manager.cc:1905] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P ec3fe6dc2f68409699ccbe96a09f7d8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:32:10.309190 11828 ts_tablet_manager.cc:1918] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P ec3fe6dc2f68409699ccbe96a09f7d8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:32:10.309355 11828 log.cc:1196] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P ec3fe6dc2f68409699ccbe96a09f7d8b: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/wal/wals/44ddf6de46cf46a3a9b2d8bfb054f6d7
I20260619 20:32:10.309707 11828 ts_tablet_manager.cc:1939] T 44ddf6de46cf46a3a9b2d8bfb054f6d7 P ec3fe6dc2f68409699ccbe96a09f7d8b: Deleting consensus metadata
W20260619 20:32:10.310287 11525 connection.cc:441] server connection from 127.0.0.1:37938 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37938 (request call id 1) could send its response
I20260619 20:32:10.311188 11787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:10.299125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49098 (local address 127.4.83.124:45667)
0619 20:32:10.299167 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:10.299169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:10.302419 (+ 3250us) server_negotiation.cc:407] Connection header received
0619 20:32:10.302530 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:10.302532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:10.302544 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:10.302573 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:10.303346 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:10.304173 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:10.305750 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:10.306077 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:10.306124 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:10.307874 (+ 1750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:10.307876 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:10.307876 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:10.307884 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:10.309647 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:10.310040 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:10.310042 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:10.310042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:10.310224 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:10.310473 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:10.310474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:10.310474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:10.310603 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:10.310697 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:10.310987 (+ 290us) server_negotiation.cc:299] Negotiation successful
0619 20:32:10.311079 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:32:10.315936 11505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:10.309016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49102 (local address 127.4.83.124:45667)
0619 20:32:10.309064 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:10.309066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:10.309206 (+ 140us) server_negotiation.cc:407] Connection header received
0619 20:32:10.309299 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:10.309300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:10.309326 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:10.309357 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:10.310003 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:10.310789 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:10.311782 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:10.312103 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:10.312146 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:10.313566 (+ 1420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:10.313567 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:10.313568 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:10.313575 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:10.314821 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:10.315133 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:10.315134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:10.315134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:10.315190 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:10.315362 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:10.315363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:10.315363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:10.315506 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:10.315609 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:10.315721 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 20:32:10.315800 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 20:32:10.318958 11789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:10.310736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52514 (local address 127.4.83.125:44301)
0619 20:32:10.310789 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:10.310791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:10.311097 (+ 306us) server_negotiation.cc:407] Connection header received
0619 20:32:10.311189 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:10.311190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:10.311205 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:10.311239 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:10.312798 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:10.313574 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:10.314649 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:10.314975 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:10.315017 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:10.316396 (+ 1379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:10.316398 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:10.316398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:10.316406 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:10.317913 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:10.318239 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:10.318240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:10.318240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:10.318300 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:10.318452 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:10.318453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:10.318453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:10.318586 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:10.318676 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:10.318777 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 20:32:10.318863 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 20:32:11.295536 11839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.295121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.4.83.66:43615)
0619 20:32:11.295413 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.295485 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":90,"threads_started":1}
I20260619 20:32:11.295536 11838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.295129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.4.83.67:43679)
0619 20:32:11.295341 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.295461 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":79,"threads_started":1}
W20260619 20:32:11.295797 11639 heartbeater.cc:644] Failed to heartbeat to 127.4.83.126:35083 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
W20260619 20:32:11.295941 11775 heartbeater.cc:644] Failed to heartbeat to 127.4.83.126:35083 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
I20260619 20:32:11.296065 11840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.295487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.4.83.65:52333)
0619 20:32:11.295899 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.296014 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":65,"threads_started":1}
W20260619 20:32:11.296296 11502 heartbeater.cc:644] Failed to heartbeat to 127.4.83.126:35083 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
I20260619 20:32:11.797673 11841 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8a05ff823034550b48979e189dfdf11)
I20260619 20:32:11.797868 11842 raft_consensus.cc:482] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c8a05ff823034550b48979e189dfdf11)
I20260619 20:32:11.797919 11841 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: false } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: false } }
I20260619 20:32:11.798040 11842 raft_consensus.cc:504] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: false } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: false } }
I20260619 20:32:11.798275 11841 leader_election.cc:288] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8a05ff823034550b48979e189dfdf11 (127.4.83.126:35083), b75ee363caf445f6844b230fb17c0e39 (127.4.83.125:44301)
I20260619 20:32:11.798353 11842 leader_election.cc:288] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8a05ff823034550b48979e189dfdf11 (127.4.83.126:35083), 3ee0d6ea97c54966bf13b731b01bbd5f (127.4.83.124:45667)
I20260619 20:32:11.798800 11843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.798512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.0.0.1:57012)
0619 20:32:11.798670 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.798751 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":75,"threads_started":1}
I20260619 20:32:11.798807 11845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.798520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.0.0.1:57004)
0619 20:32:11.798670 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.798741 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":73,"threads_started":1}
W20260619 20:32:11.798995 11122 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
W20260619 20:32:11.799043 11308 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
I20260619 20:32:11.800041 11843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.799890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.0.0.1:57036)
0619 20:32:11.799952 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.799992 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 20:32:11.800227 11122 leader_election.cc:334] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c8a05ff823034550b48979e189dfdf11 (127.4.83.126:35083): Network error: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
I20260619 20:32:11.800253 11846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.799889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.0.0.1:57024)
0619 20:32:11.800169 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.800207 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243}
W20260619 20:32:11.800348 11308 leader_election.cc:334] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c8a05ff823034550b48979e189dfdf11 (127.4.83.126:35083): Network error: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
I20260619 20:32:11.803423 11847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.798713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49118 (local address 127.4.83.124:45667)
0619 20:32:11.798989 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:11.798991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:11.799448 (+ 457us) server_negotiation.cc:407] Connection header received
0619 20:32:11.799483 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:11.799484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:11.799496 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:11.799573 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:11.799758 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:11.800564 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:11.802349 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:11.803003 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:11.803089 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:11.803108 (+ 19us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:11.803240 (+ 132us) server_negotiation.cc:299] Negotiation successful
0619 20:32:11.803306 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":63,"threads_started":1}
I20260619 20:32:11.803404 11844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.798522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:45667 (local address 127.0.0.1:49118)
0619 20:32:11.799367 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.799377 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:11.799433 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:11.799607 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:11.799608 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:11.799611 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:11.799676 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:11.799678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:11.800677 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:11.800678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:11.802186 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:11.802193 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:11.803092 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:11.803094 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:11.803169 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:11.803175 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 20:32:11.803235 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 20:32:11.803312 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":832,"thread_start_us":70,"threads_started":1}
I20260619 20:32:11.803648 11845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.798712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:44301 (local address 127.0.0.1:52518)
0619 20:32:11.798879 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.798891 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:11.798999 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:11.799370 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:11.799374 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:11.799381 (+ 7us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:11.799469 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:11.799472 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:11.800667 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:11.800669 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:11.802186 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:11.802192 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:11.803384 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:11.803386 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:11.803460 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:11.803465 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:32:11.803510 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 20:32:11.803570 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":51,"threads_started":1}
I20260619 20:32:11.803733 11331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b75ee363caf445f6844b230fb17c0e39" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" is_pre_election: true
I20260619 20:32:11.803905 11331 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b75ee363caf445f6844b230fb17c0e39 in term 1.
I20260619 20:32:11.803972 11848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.798871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52518 (local address 127.4.83.125:44301)
0619 20:32:11.799093 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:11.799094 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:11.799104 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:32:11.799141 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:11.799142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:11.799155 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:11.799291 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:11.799652 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:11.800567 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:11.802349 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:11.803311 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:11.803367 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:11.803382 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:11.803808 (+ 426us) server_negotiation.cc:299] Negotiation successful
0619 20:32:11.803898 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":64,"threads_started":1}
I20260619 20:32:11.804092 11121 leader_election.cc:302] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ee0d6ea97c54966bf13b731b01bbd5f, b75ee363caf445f6844b230fb17c0e39; no voters: c8a05ff823034550b48979e189dfdf11
I20260619 20:32:11.804224 11842 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 20:32:11.804199 11146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b75ee363caf445f6844b230fb17c0e39" is_pre_election: true
I20260619 20:32:11.804266 11842 raft_consensus.cc:482] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c8a05ff823034550b48979e189dfdf11)
I20260619 20:32:11.804325 11842 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [term 1 FOLLOWER]: Advancing to term 2
I20260619 20:32:11.805203 11842 raft_consensus.cc:504] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: false } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: false } }
I20260619 20:32:11.805404 11842 leader_election.cc:288] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [CANDIDATE]: Term 2 election: Requested vote from peers c8a05ff823034550b48979e189dfdf11 (127.4.83.126:35083), 3ee0d6ea97c54966bf13b731b01bbd5f (127.4.83.124:45667)
I20260619 20:32:11.805440 11146 raft_consensus.cc:2379] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ee0d6ea97c54966bf13b731b01bbd5f in current term 2: Already voted for candidate b75ee363caf445f6844b230fb17c0e39 in this term.
I20260619 20:32:11.805657 11844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.805548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.0.0.1:57062)
0619 20:32:11.805595 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.805629 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260619 20:32:11.805685 11331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b75ee363caf445f6844b230fb17c0e39" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f"
I20260619 20:32:11.805788 11331 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [term 1 FOLLOWER]: Advancing to term 2
I20260619 20:32:11.805948 11308 leader_election.cc:302] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ee0d6ea97c54966bf13b731b01bbd5f; no voters: b75ee363caf445f6844b230fb17c0e39, c8a05ff823034550b48979e189dfdf11
I20260619 20:32:11.805999 11844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.805876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.0.0.1:57074)
0619 20:32:11.805924 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.805979 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 20:32:11.806053 11122 leader_election.cc:334] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c8a05ff823034550b48979e189dfdf11 (127.4.83.126:35083): Network error: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
I20260619 20:32:11.806769 11331 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b75ee363caf445f6844b230fb17c0e39 in term 2.
I20260619 20:32:11.806862 11841 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 20:32:11.807000 11121 leader_election.cc:302] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ee0d6ea97c54966bf13b731b01bbd5f, b75ee363caf445f6844b230fb17c0e39; no voters: c8a05ff823034550b48979e189dfdf11
I20260619 20:32:11.807130 11842 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 20:32:11.807334 11842 raft_consensus.cc:686] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [term 2 LEADER]: Becoming Leader. State: Replica: b75ee363caf445f6844b230fb17c0e39, State: Running, Role: LEADER
I20260619 20:32:11.807466 11842 consensus_queue.cc:237] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: false } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: false } }
I20260619 20:32:11.807865 11851 sys_catalog.cc:455] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b75ee363caf445f6844b230fb17c0e39. Latest consensus state: current_term: 2 leader_uuid: "b75ee363caf445f6844b230fb17c0e39" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: false } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: false } } }
I20260619 20:32:11.807955 11851 sys_catalog.cc:458] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:11.808143 11853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:32:11.808681 11853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:32:11.808753 11853 catalog_manager.cc:1259] Loaded cluster ID: 32dbae523902416f99c2f5cbddadd76c
I20260619 20:32:11.808784 11853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:32:11.808991 11853 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:32:11.809067 11853 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39: Loaded TSK: 0
I20260619 20:32:11.809219 11853 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:32:11.862752 11136 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52514:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:32:11.864933 11136 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:32:11.866355 11844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.866171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.0.0.1:57098)
0619 20:32:11.866252 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.866306 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260619 20:32:11.866451 11331 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [term 2 FOLLOWER]: Refusing update from remote peer b75ee363caf445f6844b230fb17c0e39: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260619 20:32:11.866779 11844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.866653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.0.0.1:57104)
0619 20:32:11.866701 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.866731 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 20:32:11.866780 11851 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260619 20:32:11.866963 11122 consensus_peers.cc:487] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 -> Peer c8a05ff823034550b48979e189dfdf11 (127.4.83.126:35083): Couldn't send request to peer c8a05ff823034550b48979e189dfdf11. Status: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 20:32:11.867586 11841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b75ee363caf445f6844b230fb17c0e39. Latest consensus state: current_term: 2 leader_uuid: "b75ee363caf445f6844b230fb17c0e39" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: false } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: false } } }
I20260619 20:32:11.867692 11841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:32:11.868155 11854 mvcc.cc:205] Tried to move back new op lower bound from 7298667036122087424 to 7298667035883978752. Current Snapshot: MvccSnapshot[applied={T|T < 7298667036122087424}]
I20260619 20:32:11.868425 11841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "b75ee363caf445f6844b230fb17c0e39" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: false } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: false } } }
I20260619 20:32:11.868517 11841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ee0d6ea97c54966bf13b731b01bbd5f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:32:11.868683 11851 sys_catalog.cc:455] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "b75ee363caf445f6844b230fb17c0e39" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c8a05ff823034550b48979e189dfdf11" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35083 } } peers { permanent_uuid: "b75ee363caf445f6844b230fb17c0e39" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 44301 } attrs { promote: false } } peers { permanent_uuid: "3ee0d6ea97c54966bf13b731b01bbd5f" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 45667 } attrs { promote: false } } }
I20260619 20:32:11.868777 11851 sys_catalog.cc:458] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:11.874804 11844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.870931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:35511 (local address 127.0.0.1:48800)
0619 20:32:11.871144 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.871155 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:11.871195 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:11.871513 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:11.871514 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:11.871515 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:11.871575 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:11.871577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:11.872316 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:11.872317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:11.873606 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:11.873609 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:11.874446 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:11.874448 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:11.874535 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:11.874542 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 20:32:11.874604 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 20:32:11.874662 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":108,"threads_started":1}
I20260619 20:32:11.874787 11862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.871144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48800 (local address 127.4.83.65:35511)
0619 20:32:11.871386 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:11.871388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:11.871396 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:32:11.871424 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:11.871425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:11.871437 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:11.871501 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:11.871642 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:11.872229 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:11.873823 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:11.874358 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:11.874435 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:11.874454 (+ 19us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:11.874631 (+ 177us) server_negotiation.cc:299] Negotiation successful
0619 20:32:11.874691 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":66,"threads_started":1}
I20260619 20:32:11.874810 11843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.870829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:33863 (local address 127.0.0.1:37944)
0619 20:32:11.870904 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.870917 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:11.870976 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:11.871427 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:11.871428 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:11.871430 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:11.871474 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:11.871476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:11.872536 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:11.872537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:11.873497 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:11.873501 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:11.874491 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:11.874492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:11.874562 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:11.874566 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:32:11.874596 (+ 30us) client_negotiation.cc:240] Negotiation successful
0619 20:32:11.874660 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":90}
I20260619 20:32:11.874943 11860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.870921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34690 (local address 127.4.83.67:40087)
0619 20:32:11.871235 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:11.871237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:11.871245 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:32:11.871276 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:11.871277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:11.871291 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:11.871539 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:11.871581 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:11.872428 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:11.873802 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:11.874406 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:11.874528 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:11.874547 (+ 19us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:11.874775 (+ 228us) server_negotiation.cc:299] Negotiation successful
0619 20:32:11.874845 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":113,"threads_started":1}
I20260619 20:32:11.875051 11861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.870836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:40087 (local address 127.0.0.1:34690)
0619 20:32:11.871085 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.871093 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:11.871137 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:11.871385 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:11.871386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:11.871389 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:11.871474 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:11.871476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:11.872531 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:11.872532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:11.873712 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:11.873714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:11.874530 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:11.874531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:11.874601 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:11.874605 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:32:11.874650 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 20:32:11.874699 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":49}
I20260619 20:32:11.875325 11859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.870916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37944 (local address 127.4.83.66:33863)
0619 20:32:11.871273 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:11.871275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:11.871283 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:32:11.871314 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:11.871315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:11.871336 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:11.871537 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:11.871580 (+ 43us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:11.872427 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:11.873609 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:11.874336 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:11.874845 (+ 509us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:11.875169 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:11.875197 (+ 28us) server_negotiation.cc:299] Negotiation successful
0619 20:32:11.875260 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":105,"threads_started":1}
I20260619 20:32:11.875552 11435 tablet_service.cc:1467] Processing CreateTablet for tablet 189079daa8bd499fb827629956029c23 (DEFAULT_TABLE table=test-table [id=2668046add974fefa4f573ac00b86618]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:32:11.875571 11572 tablet_service.cc:1467] Processing CreateTablet for tablet 189079daa8bd499fb827629956029c23 (DEFAULT_TABLE table=test-table [id=2668046add974fefa4f573ac00b86618]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:32:11.875612 11708 tablet_service.cc:1467] Processing CreateTablet for tablet 189079daa8bd499fb827629956029c23 (DEFAULT_TABLE table=test-table [id=2668046add974fefa4f573ac00b86618]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:32:11.875716 11572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 189079daa8bd499fb827629956029c23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:11.875717 11435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 189079daa8bd499fb827629956029c23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:11.875722 11708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 189079daa8bd499fb827629956029c23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:11.878469 11863 tablet_bootstrap.cc:492] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b: Bootstrap starting.
I20260619 20:32:11.878824 11865 tablet_bootstrap.cc:492] T 189079daa8bd499fb827629956029c23 P 50ef078492ce4fa6a41e0f986ce8390d: Bootstrap starting.
I20260619 20:32:11.878906 11863 tablet_bootstrap.cc:654] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:11.879210 11865 tablet_bootstrap.cc:654] T 189079daa8bd499fb827629956029c23 P 50ef078492ce4fa6a41e0f986ce8390d: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:11.879961 11863 tablet_bootstrap.cc:492] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b: No bootstrap required, opened a new log
I20260619 20:32:11.879992 11864 tablet_bootstrap.cc:492] T 189079daa8bd499fb827629956029c23 P 46234eb18fcf465f96ba823f2d6753d3: Bootstrap starting.
I20260619 20:32:11.880059 11863 ts_tablet_manager.cc:1397] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:32:11.880275 11863 raft_consensus.cc:348] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33863 } } peers { permanent_uuid: "50ef078492ce4fa6a41e0f986ce8390d" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 35511 } } peers { permanent_uuid: "46234eb18fcf465f96ba823f2d6753d3" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40087 } }
I20260619 20:32:11.880366 11863 raft_consensus.cc:374] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:11.880386 11863 raft_consensus.cc:729] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec3fe6dc2f68409699ccbe96a09f7d8b, State: Initialized, Role: FOLLOWER
I20260619 20:32:11.880501 11864 tablet_bootstrap.cc:654] T 189079daa8bd499fb827629956029c23 P 46234eb18fcf465f96ba823f2d6753d3: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:11.880470 11863 consensus_queue.cc:260] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33863 } } peers { permanent_uuid: "50ef078492ce4fa6a41e0f986ce8390d" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 35511 } } peers { permanent_uuid: "46234eb18fcf465f96ba823f2d6753d3" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40087 } }
I20260619 20:32:11.880815 11863 ts_tablet_manager.cc:1428] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:11.881311 11864 tablet_bootstrap.cc:492] T 189079daa8bd499fb827629956029c23 P 46234eb18fcf465f96ba823f2d6753d3: No bootstrap required, opened a new log
I20260619 20:32:11.881371 11864 ts_tablet_manager.cc:1397] T 189079daa8bd499fb827629956029c23 P 46234eb18fcf465f96ba823f2d6753d3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:11.881508 11864 raft_consensus.cc:348] T 189079daa8bd499fb827629956029c23 P 46234eb18fcf465f96ba823f2d6753d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33863 } } peers { permanent_uuid: "50ef078492ce4fa6a41e0f986ce8390d" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 35511 } } peers { permanent_uuid: "46234eb18fcf465f96ba823f2d6753d3" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40087 } }
I20260619 20:32:11.881618 11864 raft_consensus.cc:374] T 189079daa8bd499fb827629956029c23 P 46234eb18fcf465f96ba823f2d6753d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:11.881634 11864 raft_consensus.cc:729] T 189079daa8bd499fb827629956029c23 P 46234eb18fcf465f96ba823f2d6753d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46234eb18fcf465f96ba823f2d6753d3, State: Initialized, Role: FOLLOWER
I20260619 20:32:11.881676 11864 consensus_queue.cc:260] T 189079daa8bd499fb827629956029c23 P 46234eb18fcf465f96ba823f2d6753d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33863 } } peers { permanent_uuid: "50ef078492ce4fa6a41e0f986ce8390d" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 35511 } } peers { permanent_uuid: "46234eb18fcf465f96ba823f2d6753d3" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40087 } }
I20260619 20:32:11.881805 11870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.881200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.4.83.66:34935)
0619 20:32:11.881700 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.881754 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":234,"threads_started":1}
I20260619 20:32:11.881827 11864 ts_tablet_manager.cc:1428] T 189079daa8bd499fb827629956029c23 P 46234eb18fcf465f96ba823f2d6753d3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 20:32:11.882304 11870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.882166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.4.83.66:57115)
0619 20:32:11.882213 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.882237 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 20:32:11.881927 11868 raft_consensus.cc:482] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:32:11.882426 11868 raft_consensus.cc:504] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33863 } } peers { permanent_uuid: "50ef078492ce4fa6a41e0f986ce8390d" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 35511 } } peers { permanent_uuid: "46234eb18fcf465f96ba823f2d6753d3" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40087 } }
I20260619 20:32:11.882510 11640 heartbeater.cc:503] Master 127.4.83.125:44301 was elected leader, sending a full tablet report...
I20260619 20:32:11.882738 11865 tablet_bootstrap.cc:492] T 189079daa8bd499fb827629956029c23 P 50ef078492ce4fa6a41e0f986ce8390d: No bootstrap required, opened a new log
I20260619 20:32:11.882815 11865 ts_tablet_manager.cc:1397] T 189079daa8bd499fb827629956029c23 P 50ef078492ce4fa6a41e0f986ce8390d: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 20:32:11.883002 11865 raft_consensus.cc:348] T 189079daa8bd499fb827629956029c23 P 50ef078492ce4fa6a41e0f986ce8390d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33863 } } peers { permanent_uuid: "50ef078492ce4fa6a41e0f986ce8390d" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 35511 } } peers { permanent_uuid: "46234eb18fcf465f96ba823f2d6753d3" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40087 } }
I20260619 20:32:11.883087 11865 raft_consensus.cc:374] T 189079daa8bd499fb827629956029c23 P 50ef078492ce4fa6a41e0f986ce8390d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:11.883108 11865 raft_consensus.cc:729] T 189079daa8bd499fb827629956029c23 P 50ef078492ce4fa6a41e0f986ce8390d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50ef078492ce4fa6a41e0f986ce8390d, State: Initialized, Role: FOLLOWER
I20260619 20:32:11.883322 11865 consensus_queue.cc:260] T 189079daa8bd499fb827629956029c23 P 50ef078492ce4fa6a41e0f986ce8390d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33863 } } peers { permanent_uuid: "50ef078492ce4fa6a41e0f986ce8390d" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 35511 } } peers { permanent_uuid: "46234eb18fcf465f96ba823f2d6753d3" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40087 } }
I20260619 20:32:11.883806 11868 leader_election.cc:288] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50ef078492ce4fa6a41e0f986ce8390d (127.4.83.65:35511), 46234eb18fcf465f96ba823f2d6753d3 (127.4.83.67:40087)
I20260619 20:32:11.883530 11865 ts_tablet_manager.cc:1428] T 189079daa8bd499fb827629956029c23 P 50ef078492ce4fa6a41e0f986ce8390d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:11.884406 11503 heartbeater.cc:503] Master 127.4.83.125:44301 was elected leader, sending a full tablet report...
I20260619 20:32:11.884727 11876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.883684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.4.83.65:39035)
0619 20:32:11.884625 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.884673 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":915,"thread_start_us":81,"threads_started":1}
I20260619 20:32:11.885177 11873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.882211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.4.83.67:47177)
0619 20:32:11.885073 (+ 2862us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.885127 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2828,"thread_start_us":108,"threads_started":1}
I20260619 20:32:11.885390 11876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.885055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.4.83.65:52919)
0619 20:32:11.885335 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.885365 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264}
I20260619 20:32:11.885834 11776 heartbeater.cc:503] Master 127.4.83.125:44301 was elected leader, sending a full tablet report...
I20260619 20:32:11.886034 11873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.885672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.4.83.67:54975)
0619 20:32:11.885976 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.886006 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":101,"threads_started":1}
I20260619 20:32:11.887241 11862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.883128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:46679 (local address 127.4.83.65:35511)
0619 20:32:11.883184 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:11.883186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:11.883705 (+ 519us) server_negotiation.cc:407] Connection header received
0619 20:32:11.883797 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:11.883798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:11.883809 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:11.883892 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:11.884079 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:11.884893 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:11.886514 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:11.886925 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:11.886980 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:11.886994 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:11.887143 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 20:32:11.887183 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 20:32:11.887260 11860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.883337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:38119 (local address 127.4.83.67:40087)
0619 20:32:11.883421 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:11.883423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:11.883432 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:32:11.883544 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:11.883545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:11.883562 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:11.883592 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:11.884288 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:11.884979 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:11.886545 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:11.886941 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:11.887005 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:11.887018 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:11.887143 (+ 125us) server_negotiation.cc:299] Negotiation successful
0619 20:32:11.887196 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 20:32:11.887285 11870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.883044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:35511 (local address 127.4.83.66:46679)
0619 20:32:11.883494 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.883504 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:11.883730 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:11.883941 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:11.883942 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:11.883944 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:11.884001 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:11.884003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:11.885017 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:11.885018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:11.886418 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:11.886421 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:11.887013 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:11.887014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:11.887079 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:11.887083 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:32:11.887128 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 20:32:11.887202 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"mutex_wait_us":140,"thread_start_us":113,"threads_started":1}
I20260619 20:32:11.887549 11455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "189079daa8bd499fb827629956029c23" candidate_uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ef078492ce4fa6a41e0f986ce8390d" is_pre_election: true
I20260619 20:32:11.887643 11455 raft_consensus.cc:2454] T 189079daa8bd499fb827629956029c23 P 50ef078492ce4fa6a41e0f986ce8390d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec3fe6dc2f68409699ccbe96a09f7d8b in term 0.
I20260619 20:32:11.887658 11874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.883044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:40087 (local address 127.4.83.66:38119)
0619 20:32:11.883348 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.883360 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:11.883455 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:11.883670 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:11.883671 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:11.883673 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:11.884211 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:11.884214 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:11.885115 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:11.885117 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:11.886445 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:11.886448 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:11.887018 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:11.887019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:11.887081 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:11.887084 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:32:11.887129 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 20:32:11.887193 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":3}
I20260619 20:32:11.887812 11525 leader_election.cc:302] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50ef078492ce4fa6a41e0f986ce8390d, ec3fe6dc2f68409699ccbe96a09f7d8b; no voters:
I20260619 20:32:11.888007 11872 raft_consensus.cc:2781] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:32:11.887928 11728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "189079daa8bd499fb827629956029c23" candidate_uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46234eb18fcf465f96ba823f2d6753d3" is_pre_election: true
I20260619 20:32:11.888056 11872 raft_consensus.cc:482] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:32:11.888082 11872 raft_consensus.cc:3037] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:11.888091 11728 raft_consensus.cc:2454] T 189079daa8bd499fb827629956029c23 P 46234eb18fcf465f96ba823f2d6753d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec3fe6dc2f68409699ccbe96a09f7d8b in term 0.
I20260619 20:32:11.888991 11872 raft_consensus.cc:504] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33863 } } peers { permanent_uuid: "50ef078492ce4fa6a41e0f986ce8390d" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 35511 } } peers { permanent_uuid: "46234eb18fcf465f96ba823f2d6753d3" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40087 } }
I20260619 20:32:11.889129 11872 leader_election.cc:288] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b [CANDIDATE]: Term 1 election: Requested vote from peers 50ef078492ce4fa6a41e0f986ce8390d (127.4.83.65:35511), 46234eb18fcf465f96ba823f2d6753d3 (127.4.83.67:40087)
I20260619 20:32:11.889351 11728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "189079daa8bd499fb827629956029c23" candidate_uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46234eb18fcf465f96ba823f2d6753d3"
I20260619 20:32:11.889436 11728 raft_consensus.cc:3037] T 189079daa8bd499fb827629956029c23 P 46234eb18fcf465f96ba823f2d6753d3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:11.889487 11455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "189079daa8bd499fb827629956029c23" candidate_uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ef078492ce4fa6a41e0f986ce8390d"
I20260619 20:32:11.889562 11455 raft_consensus.cc:3037] T 189079daa8bd499fb827629956029c23 P 50ef078492ce4fa6a41e0f986ce8390d [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:11.890467 11455 raft_consensus.cc:2454] T 189079daa8bd499fb827629956029c23 P 50ef078492ce4fa6a41e0f986ce8390d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec3fe6dc2f68409699ccbe96a09f7d8b in term 1.
I20260619 20:32:11.890455 11728 raft_consensus.cc:2454] T 189079daa8bd499fb827629956029c23 P 46234eb18fcf465f96ba823f2d6753d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec3fe6dc2f68409699ccbe96a09f7d8b in term 1.
I20260619 20:32:11.890738 11526 leader_election.cc:302] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 46234eb18fcf465f96ba823f2d6753d3, 50ef078492ce4fa6a41e0f986ce8390d, ec3fe6dc2f68409699ccbe96a09f7d8b; no voters:
I20260619 20:32:11.890887 11872 raft_consensus.cc:2781] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:32:11.890961 11872 raft_consensus.cc:686] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b [term 1 LEADER]: Becoming Leader. State: Replica: ec3fe6dc2f68409699ccbe96a09f7d8b, State: Running, Role: LEADER
I20260619 20:32:11.891153 11872 consensus_queue.cc:237] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33863 } } peers { permanent_uuid: "50ef078492ce4fa6a41e0f986ce8390d" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 35511 } } peers { permanent_uuid: "46234eb18fcf465f96ba823f2d6753d3" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40087 } }
I20260619 20:32:11.891428 11874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.891262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.4.83.66:33809)
0619 20:32:11.891323 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.891385 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 20:32:11.892238 11136 catalog_manager.cc:5495] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b reported cstate change: term changed from 0 to 1, leader changed from <none> to ec3fe6dc2f68409699ccbe96a09f7d8b (127.4.83.66). New cstate: current_term: 1 leader_uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3fe6dc2f68409699ccbe96a09f7d8b" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50ef078492ce4fa6a41e0f986ce8390d" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 35511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46234eb18fcf465f96ba823f2d6753d3" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40087 } health_report { overall_health: UNKNOWN } } }
I20260619 20:32:11.898972 11859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.895820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37950 (local address 127.4.83.66:33863)
0619 20:32:11.895889 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:11.895891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:11.895900 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:32:11.895987 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:11.895988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:11.896011 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:11.896067 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:32:11.896174 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:11.896791 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:11.898196 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:11.898440 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:11.898505 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:11.898637 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:32:11.898765 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:32:11.898828 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:11.898859 (+ 31us) server_negotiation.cc:299] Negotiation successful
0619 20:32:11.898883 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 20:32:11.900535 11455 raft_consensus.cc:1261] T 189079daa8bd499fb827629956029c23 P 50ef078492ce4fa6a41e0f986ce8390d [term 1 FOLLOWER]: Refusing update from remote peer ec3fe6dc2f68409699ccbe96a09f7d8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:32:11.900702 11728 raft_consensus.cc:1261] T 189079daa8bd499fb827629956029c23 P 46234eb18fcf465f96ba823f2d6753d3 [term 1 FOLLOWER]: Refusing update from remote peer ec3fe6dc2f68409699ccbe96a09f7d8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:32:11.901067 11872 consensus_queue.cc:1035] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "50ef078492ce4fa6a41e0f986ce8390d" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 35511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:32:11.901396 11868 consensus_queue.cc:1035] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "46234eb18fcf465f96ba823f2d6753d3" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:32:11.902243 11876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.902055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.4.83.65:46275)
0619 20:32:11.902136 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.902180 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 20:32:11.903033 11877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.902732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.4.83.67:57925)
0619 20:32:11.902935 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.902988 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172}
I20260619 20:32:11.904327 11876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.904220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.4.83.65:53089)
0619 20:32:11.904257 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.904287 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260619 20:32:11.905040 11874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.904888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.4.83.66:40415)
0619 20:32:11.904972 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.905001 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260619 20:32:11.905192 11877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.905054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35083 (local address 127.4.83.67:46305)
0619 20:32:11.905125 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.905153 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260619 20:32:11.907563 11136 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:32:11.907713 11136 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:32:11.909462 11136 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b75ee363caf445f6844b230fb17c0e39: Sending DeleteTablet for 3 replicas of tablet 189079daa8bd499fb827629956029c23
I20260619 20:32:11.909854 11708 tablet_service.cc:1514] Processing DeleteTablet for tablet 189079daa8bd499fb827629956029c23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:32:11 UTC) from {username='oryx', principal='oryx/127.4.83.125@KRBTEST.COM'} at 127.0.0.1:34690
I20260619 20:32:11.909976 11435 tablet_service.cc:1514] Processing DeleteTablet for tablet 189079daa8bd499fb827629956029c23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:32:11 UTC) from {username='oryx', principal='oryx/127.4.83.125@KRBTEST.COM'} at 127.0.0.1:48800
I20260619 20:32:11.910138 11890 tablet_replica.cc:330] T 189079daa8bd499fb827629956029c23 P 46234eb18fcf465f96ba823f2d6753d3: stopping tablet replica
I20260619 20:32:11.910202 11890 raft_consensus.cc:2229] T 189079daa8bd499fb827629956029c23 P 46234eb18fcf465f96ba823f2d6753d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:32:11.910264 11890 raft_consensus.cc:2258] T 189079daa8bd499fb827629956029c23 P 46234eb18fcf465f96ba823f2d6753d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:32:11.910347 11572 tablet_service.cc:1514] Processing DeleteTablet for tablet 189079daa8bd499fb827629956029c23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:32:11 UTC) from {username='oryx', principal='oryx/127.4.83.125@KRBTEST.COM'} at 127.0.0.1:37944
I20260619 20:32:11.910642 11891 tablet_replica.cc:330] T 189079daa8bd499fb827629956029c23 P 50ef078492ce4fa6a41e0f986ce8390d: stopping tablet replica
I20260619 20:32:11.910743 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 11283
I20260619 20:32:11.910784 11892 tablet_replica.cc:330] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b: stopping tablet replica
I20260619 20:32:11.910902 11892 raft_consensus.cc:2229] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:32:11.911002 11892 raft_consensus.cc:2258] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:32:11.911078 11891 raft_consensus.cc:2229] T 189079daa8bd499fb827629956029c23 P 50ef078492ce4fa6a41e0f986ce8390d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:32:11.911134 11891 raft_consensus.cc:2258] T 189079daa8bd499fb827629956029c23 P 50ef078492ce4fa6a41e0f986ce8390d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:32:11.911878 11892 ts_tablet_manager.cc:1905] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:32:11.913693 11892 ts_tablet_manager.cc:1918] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:32:11.913771 11892 log.cc:1196] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-1/wal/wals/189079daa8bd499fb827629956029c23
I20260619 20:32:11.914036 11892 ts_tablet_manager.cc:1939] T 189079daa8bd499fb827629956029c23 P ec3fe6dc2f68409699ccbe96a09f7d8b: Deleting consensus metadata
I20260619 20:32:11.914474 11120 catalog_manager.cc:4841] TS ec3fe6dc2f68409699ccbe96a09f7d8b (127.4.83.66:33863): tablet 189079daa8bd499fb827629956029c23 (table test-table [id=2668046add974fefa4f573ac00b86618]) successfully deleted
I20260619 20:32:11.916301 11890 ts_tablet_manager.cc:1905] T 189079daa8bd499fb827629956029c23 P 46234eb18fcf465f96ba823f2d6753d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:32:11.918165 11890 ts_tablet_manager.cc:1918] T 189079daa8bd499fb827629956029c23 P 46234eb18fcf465f96ba823f2d6753d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:32:11.918267 11890 log.cc:1196] T 189079daa8bd499fb827629956029c23 P 46234eb18fcf465f96ba823f2d6753d3: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901102012824-4429-0/minicluster-data/ts-2/wal/wals/189079daa8bd499fb827629956029c23
I20260619 20:32:11.918514 11890 ts_tablet_manager.cc:1939] T 189079daa8bd499fb827629956029c23 P 46234eb18fcf465f96ba823f2d6753d3: Deleting consensus metadata
I20260619 20:32:11.919024 11121 catalog_manager.cc:4841] TS 46234eb18fcf465f96ba823f2d6753d3 (127.4.83.67:40087): tablet 189079daa8bd499fb827629956029c23 (table test-table [id=2668046add974fefa4f573ac00b86618]) successfully deleted
I20260619 20:32:11.920077 11843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:11.919933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:35511 (local address 127.0.0.1:48814)
0619 20:32:11.919998 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:11.920030 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.65:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 20:32:11.920212 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 11509
W20260619 20:32:11.920284 11120 catalog_manager.cc:4584] TS 50ef078492ce4fa6a41e0f986ce8390d (127.4.83.65:35511): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 189079daa8bd499fb827629956029c23: Network error: Client connection negotiation failed: client connection to 127.4.83.65:35511: connect: Connection refused (error 111)
I20260619 20:32:11.927735 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 11645
I20260619 20:32:11.933773 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 11002
I20260619 20:32:11.939167 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 11190
2026-06-19T20:32:11Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4675 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:32:12 dist-test-slave-4m58 krb5kdc[11895](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:32:12 dist-test-slave-4m58 krb5kdc[11895](info): set up 2 sockets
Jun 19 20:32:12 dist-test-slave-4m58 krb5kdc[11895](info): commencing operation
krb5kdc: starting...
W20260619 20:32:12.937036 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.975s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:32:13 dist-test-slave-4m58 krb5kdc[11895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:32:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:32:13Z Disabled control of system clock
WARNING: no policy specified for oryx/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.126.keytab.
Entry for principal oryx/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.126.keytab.
I20260619 20:32:13.071583 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:43281
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.126.keytab
--principal=oryx/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:38389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:43281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:13.154232 11908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:13.154488 11908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:13.154531 11908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:13.155762 11908 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:32:13.155821 11908 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 20:32:13.155894 11908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:13.155915 11908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:13.155948 11908 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:32:13.155972 11908 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:32:13.157898 11908 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:38389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:43281
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.126.keytab
--principal=oryx/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:43281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:13.158197 11908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:13.158420 11908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:13.163471 11916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:13.163517 11914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:13.163570 11908 server_base.cc:1034] running on GCE node
W20260619 20:32:13.163488 11913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:13.164013 11908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:13.164331 11908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:13.165513 11908 hybrid_clock.cc:648] HybridClock initialized: now 1781901133165479 us; error 54 us; skew 500 ppm
Jun 19 20:32:13 dist-test-slave-4m58 krb5kdc[11895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901133, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:13.168102 11908 init.cc:377] Logged in from keytab as oryx/127.4.83.126@KRBTEST.COM (short username oryx)
I20260619 20:32:13.168555 11908 webserver.cc:460] Webserver started at http://127.4.83.126:44565/ using document root <none> and password file <none>
I20260619 20:32:13.168759 11908 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:13.168800 11908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:13.168892 11908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:13.169909 11908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "74439e8536d84e1884988644eba75b84"
format_stamp: "Formatted at 2026-06-19 20:32:13 on dist-test-slave-4m58"
server_key: "85397f85efc02ab47974430d46add179"
server_key_iv: "112d6abfe86be5e3e4db409273299fc8"
server_key_version: "encryptionkey@0"
I20260619 20:32:13.170203 11908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "74439e8536d84e1884988644eba75b84"
format_stamp: "Formatted at 2026-06-19 20:32:13 on dist-test-slave-4m58"
server_key: "85397f85efc02ab47974430d46add179"
server_key_iv: "112d6abfe86be5e3e4db409273299fc8"
server_key_version: "encryptionkey@0"
I20260619 20:32:13.171507 11908 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:32:13.172304 11923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:13.172458 11908 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:13.172542 11908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "74439e8536d84e1884988644eba75b84"
format_stamp: "Formatted at 2026-06-19 20:32:13 on dist-test-slave-4m58"
server_key: "85397f85efc02ab47974430d46add179"
server_key_iv: "112d6abfe86be5e3e4db409273299fc8"
server_key_version: "encryptionkey@0"
I20260619 20:32:13.172641 11908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:13.212301 11908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:13.214592 11908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:13.214743 11908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:13.219277 11908 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:43281
I20260619 20:32:13.219326 11985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:43281 every 8 connection(s)
I20260619 20:32:13.219658 11908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:32:13.220340 11986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:13.222661 11986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74439e8536d84e1884988644eba75b84: Bootstrap starting.
I20260619 20:32:13.223251 11986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74439e8536d84e1884988644eba75b84: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:13.223569 11986 log.cc:824] T 00000000000000000000000000000000 P 74439e8536d84e1884988644eba75b84: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:13.224396 11986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74439e8536d84e1884988644eba75b84: No bootstrap required, opened a new log
I20260619 20:32:13.227185 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 11908
I20260619 20:32:13.227099 11986 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74439e8536d84e1884988644eba75b84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74439e8536d84e1884988644eba75b84" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 43281 } }
I20260619 20:32:13.227290 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:32:13.227289 11986 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74439e8536d84e1884988644eba75b84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:13.227326 11986 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74439e8536d84e1884988644eba75b84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74439e8536d84e1884988644eba75b84, State: Initialized, Role: FOLLOWER
I20260619 20:32:13.227505 4429 external_mini_cluster.cc:1428] Setting key af1355afc5ea009e535e69276c87fb53
I20260619 20:32:13.227481 11986 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74439e8536d84e1884988644eba75b84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74439e8536d84e1884988644eba75b84" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 43281 } }
I20260619 20:32:13.227571 11986 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74439e8536d84e1884988644eba75b84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:32:13.227609 11986 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74439e8536d84e1884988644eba75b84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:32:13.227658 11986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74439e8536d84e1884988644eba75b84 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:13.228386 11986 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74439e8536d84e1884988644eba75b84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74439e8536d84e1884988644eba75b84" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 43281 } }
I20260619 20:32:13.228488 11986 leader_election.cc:302] T 00000000000000000000000000000000 P 74439e8536d84e1884988644eba75b84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74439e8536d84e1884988644eba75b84; no voters:
I20260619 20:32:13.228688 11986 leader_election.cc:288] T 00000000000000000000000000000000 P 74439e8536d84e1884988644eba75b84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:32:13.228729 11991 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74439e8536d84e1884988644eba75b84 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:32:13.228824 11991 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74439e8536d84e1884988644eba75b84 [term 1 LEADER]: Becoming Leader. State: Replica: 74439e8536d84e1884988644eba75b84, State: Running, Role: LEADER
I20260619 20:32:13.228952 11986 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74439e8536d84e1884988644eba75b84 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:32:13.228964 11991 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74439e8536d84e1884988644eba75b84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74439e8536d84e1884988644eba75b84" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 43281 } }
I20260619 20:32:13.230293 11993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74439e8536d84e1884988644eba75b84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74439e8536d84e1884988644eba75b84. Latest consensus state: current_term: 1 leader_uuid: "74439e8536d84e1884988644eba75b84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74439e8536d84e1884988644eba75b84" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 43281 } } }
I20260619 20:32:13.230389 11993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74439e8536d84e1884988644eba75b84 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:13.230612 11992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74439e8536d84e1884988644eba75b84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74439e8536d84e1884988644eba75b84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74439e8536d84e1884988644eba75b84" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 43281 } } }
I20260619 20:32:13.230698 11992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74439e8536d84e1884988644eba75b84 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:13.230770 12006 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:32:13.231267 12006 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:32:13.233053 12006 catalog_manager.cc:1347] Generated new cluster ID: 6aab88e20bb44e239e92084741da37c4
I20260619 20:32:13.233099 12006 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:32:13 dist-test-slave-4m58 krb5kdc[11895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM
I20260619 20:32:13.239718 11990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:13.227992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59882 (local address 127.4.83.126:43281)
0619 20:32:13.228201 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:13.228210 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:13.228237 (+ 27us) server_negotiation.cc:407] Connection header received
0619 20:32:13.228332 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:13.228337 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:13.228380 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:13.228494 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:13.229241 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.230560 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.231395 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.231801 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.231920 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:13.234725 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:13.234729 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:13.234729 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:13.234742 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:13.237536 (+ 2794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:13.237983 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:13.237987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:13.237988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:13.238130 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:13.238339 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:13.238340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:13.238340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:13.238670 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:13.238855 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:13.239334 (+ 479us) server_negotiation.cc:299] Negotiation successful
0619 20:32:13.239482 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":100,"threads_started":1}
I20260619 20:32:13.253399 12006 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:32:13.253955 12006 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:32:13.257241 12006 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74439e8536d84e1884988644eba75b84: Generated new TSK 0
I20260619 20:32:13.257441 12006 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.4.83.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.83.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.83.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.65.keytab.
Entry for principal oryx/127.4.83.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.65.keytab.
I20260619 20:32:13.287166 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:0
--local_ip_for_outbound_sockets=127.4.83.65
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:43281
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.65.keytab
--principal=oryx/127.4.83.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:38389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:13.375617 12012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:13.375876 12012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:13.375916 12012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:13.377238 12012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:13.377293 12012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:13.377324 12012 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 20:32:13.377349 12012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:32:13.378993 12012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:38389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.65.keytab
--principal=oryx/127.4.83.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.83.126:43281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:13.379268 12012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:13.379493 12012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:13.384538 12017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:13.384532 12018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:13.384569 12012 server_base.cc:1034] running on GCE node
W20260619 20:32:13.384532 12020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:13.384960 12012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:13.385175 12012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:13.386333 12012 hybrid_clock.cc:648] HybridClock initialized: now 1781901133386322 us; error 29 us; skew 500 ppm
Jun 19 20:32:13 dist-test-slave-4m58 krb5kdc[11895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901133, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:13.388923 12012 init.cc:377] Logged in from keytab as oryx/127.4.83.65@KRBTEST.COM (short username oryx)
I20260619 20:32:13.389346 12012 webserver.cc:460] Webserver started at http://127.4.83.65:34995/ using document root <none> and password file <none>
I20260619 20:32:13.389559 12012 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:13.389605 12012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:13.389734 12012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:13.390776 12012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-0/data/instance:
uuid: "9db5cca45bf244168b2aa2cfd5454764"
format_stamp: "Formatted at 2026-06-19 20:32:13 on dist-test-slave-4m58"
server_key: "38e786a7b975564e037dbd0308c34b25"
server_key_iv: "953f6d0275dc023ff9e8ae41de2ec920"
server_key_version: "encryptionkey@0"
I20260619 20:32:13.391103 12012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance:
uuid: "9db5cca45bf244168b2aa2cfd5454764"
format_stamp: "Formatted at 2026-06-19 20:32:13 on dist-test-slave-4m58"
server_key: "38e786a7b975564e037dbd0308c34b25"
server_key_iv: "953f6d0275dc023ff9e8ae41de2ec920"
server_key_version: "encryptionkey@0"
I20260619 20:32:13.392405 12012 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:32:13.393189 12027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:13.393329 12012 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:32:13.393412 12012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "9db5cca45bf244168b2aa2cfd5454764"
format_stamp: "Formatted at 2026-06-19 20:32:13 on dist-test-slave-4m58"
server_key: "38e786a7b975564e037dbd0308c34b25"
server_key_iv: "953f6d0275dc023ff9e8ae41de2ec920"
server_key_version: "encryptionkey@0"
I20260619 20:32:13.393476 12012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:13.407940 12012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:13.410071 12012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:13.410214 12012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:13.410701 12012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:13.410741 12012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:13.410775 12012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:13.410799 12012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 20:32:13 dist-test-slave-4m58 krb5kdc[11895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901133, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM
I20260619 20:32:13.417862 12012 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:39085
I20260619 20:32:13.418198 12012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
I20260619 20:32:13.418201 12143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:39085 every 8 connection(s)
I20260619 20:32:13.420437 11990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:13.411259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:34401 (local address 127.4.83.126:43281)
0619 20:32:13.411303 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:13.411305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:13.411653 (+ 348us) server_negotiation.cc:407] Connection header received
0619 20:32:13.412369 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:13.412371 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:13.412386 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:13.412449 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:13.413349 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.413896 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.414891 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.415120 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.415176 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:13.417700 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:13.417701 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:13.417702 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:13.417711 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:13.419124 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:13.419496 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:13.419498 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:13.419498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:13.419554 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:13.419774 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:13.419775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:13.419775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:13.419942 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:13.420001 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:13.420255 (+ 254us) server_negotiation.cc:299] Negotiation successful
0619 20:32:13.420347 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:32:13.420591 12036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:13.411290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:43281 (local address 127.4.83.65:34401)
0619 20:32:13.411520 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:13.411567 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:13.412264 (+ 697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:13.412590 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:13.412594 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:13.412943 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:13.413244 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:13.413251 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.413984 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.413985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:13.414804 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:13.414807 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.415200 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.415201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:13.415309 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:13.415858 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:13.415877 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:13.417592 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:13.419201 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:13.419206 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:13.419208 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:13.419420 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:13.419609 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:13.419610 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:13.419610 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:13.419719 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:13.419997 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:13.420003 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:13.420139 (+ 136us) client_negotiation.cc:769] Sending connection context
0619 20:32:13.420226 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 20:32:13.420383 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":74,"threads_started":1}
I20260619 20:32:13.421069 12144 heartbeater.cc:347] Connected to a master server at 127.4.83.126:43281
I20260619 20:32:13.421166 12144 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:13.421332 12144 heartbeater.cc:511] Master 127.4.83.126:43281 requested a full tablet report, sending...
I20260619 20:32:13.421839 11940 ts_manager.cc:194] Registered new tserver with Master: 9db5cca45bf244168b2aa2cfd5454764 (127.4.83.65:39085)
I20260619 20:32:13.422600 11940 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.83.65@KRBTEST.COM'} at 127.4.83.65:34401
I20260619 20:32:13.422662 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 12012
I20260619 20:32:13.422745 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance
I20260619 20:32:13.422932 4429 external_mini_cluster.cc:1428] Setting key 12cdac8d935f7c642957972922e9610f
I20260619 20:32:13.428341 11990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:13.422066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:41939 (local address 127.4.83.126:43281)
0619 20:32:13.422106 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:13.422108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:13.422171 (+ 63us) server_negotiation.cc:407] Connection header received
0619 20:32:13.422218 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:13.422219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:13.422228 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:13.422288 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:13.422566 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.423223 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.424042 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.424297 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.424369 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:13.425417 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:13.425418 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:13.425419 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:13.425425 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:13.426918 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:13.427374 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:13.427375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:13.427375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:13.427439 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:13.427672 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:13.427673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:13.427674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:13.427934 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:13.428004 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:13.428183 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 20:32:13.428270 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:32:13.428400 12150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:13.422008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:43281 (local address 127.4.83.65:41939)
0619 20:32:13.422121 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:13.422130 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:13.422182 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:13.422291 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:13.422292 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:13.422442 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:13.422503 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:13.422505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.423293 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.423293 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:13.423972 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:13.423974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.424392 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.424393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:13.424452 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:13.424904 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:13.424909 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:13.425317 (+ 408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:13.427025 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:13.427026 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:13.427027 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:13.427273 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:13.427511 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:13.427512 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:13.427513 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:13.427590 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:13.428011 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:13.428012 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:13.428079 (+ 67us) client_negotiation.cc:769] Sending connection context
0619 20:32:13.428162 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 20:32:13.428284 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":66,"threads_started":1}
WARNING: no policy specified for oryx/127.4.83.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.83.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.83.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.66.keytab.
Entry for principal oryx/127.4.83.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.66.keytab.
I20260619 20:32:13.450547 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:0
--local_ip_for_outbound_sockets=127.4.83.66
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:43281
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.66.keytab
--principal=oryx/127.4.83.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:38389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:13.532815 12153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:13.533046 12153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:13.533079 12153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:13.534271 12153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:13.534320 12153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:13.534350 12153 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 20:32:13.534375 12153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:32:13.536010 12153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:38389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.66.keytab
--principal=oryx/127.4.83.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.83.126:43281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:13.536259 12153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:13.536477 12153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:13.541795 12159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:13.541926 12153 server_base.cc:1034] running on GCE node
W20260619 20:32:13.541795 12161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:13.541797 12158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:13.542296 12153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:13.542539 12153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:13.543710 12153 hybrid_clock.cc:648] HybridClock initialized: now 1781901133543703 us; error 37 us; skew 500 ppm
Jun 19 20:32:13 dist-test-slave-4m58 krb5kdc[11895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901133, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:13.550268 12153 init.cc:377] Logged in from keytab as oryx/127.4.83.66@KRBTEST.COM (short username oryx)
I20260619 20:32:13.550799 12153 webserver.cc:460] Webserver started at http://127.4.83.66:33443/ using document root <none> and password file <none>
I20260619 20:32:13.551007 12153 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:13.551055 12153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:13.551185 12153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:13.552297 12153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-1/data/instance:
uuid: "60525532c20a4c358974dc8e75b0596d"
format_stamp: "Formatted at 2026-06-19 20:32:13 on dist-test-slave-4m58"
server_key: "fc6700157e2fb2212051c7a62627cef5"
server_key_iv: "9f739b8cc3bdce333f38e6c09de5e970"
server_key_version: "encryptionkey@0"
I20260619 20:32:13.552629 12153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance:
uuid: "60525532c20a4c358974dc8e75b0596d"
format_stamp: "Formatted at 2026-06-19 20:32:13 on dist-test-slave-4m58"
server_key: "fc6700157e2fb2212051c7a62627cef5"
server_key_iv: "9f739b8cc3bdce333f38e6c09de5e970"
server_key_version: "encryptionkey@0"
I20260619 20:32:13.553963 12153 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:32:13.554739 12168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:13.554930 12153 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:13.555018 12153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "60525532c20a4c358974dc8e75b0596d"
format_stamp: "Formatted at 2026-06-19 20:32:13 on dist-test-slave-4m58"
server_key: "fc6700157e2fb2212051c7a62627cef5"
server_key_iv: "9f739b8cc3bdce333f38e6c09de5e970"
server_key_version: "encryptionkey@0"
I20260619 20:32:13.555090 12153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:13.568638 12153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:13.570588 12153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:13.570729 12153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:13.571321 12153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:13.571370 12153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:13.571398 12153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:13.571414 12153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 20:32:13 dist-test-slave-4m58 krb5kdc[11895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901133, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM
I20260619 20:32:13.578240 12153 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:40003
I20260619 20:32:13.578567 12283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:40003 every 8 connection(s)
I20260619 20:32:13.578676 12153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
I20260619 20:32:13.580688 11990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:13.571718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:56743 (local address 127.4.83.126:43281)
0619 20:32:13.571769 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:13.571771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:13.571990 (+ 219us) server_negotiation.cc:407] Connection header received
0619 20:32:13.572526 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:13.572528 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:13.572542 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:13.572611 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:13.573299 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.573812 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.574804 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.575036 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.575094 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:13.577997 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:13.577999 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:13.577999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:13.578009 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:13.579415 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:13.579889 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:13.579891 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:13.579891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:13.579958 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:13.580190 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:13.580190 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:13.580190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:13.580299 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:13.580346 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:13.580556 (+ 210us) server_negotiation.cc:299] Negotiation successful
0619 20:32:13.580623 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 20:32:13.580849 12177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:13.571696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:43281 (local address 127.4.83.66:56743)
0619 20:32:13.571862 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:13.571914 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:13.572427 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:13.572683 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:13.572688 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:13.572990 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:13.573206 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:13.573211 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.573888 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.573889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:13.574717 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:13.574720 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.575113 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.575115 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:13.575265 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:13.575881 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:13.575891 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:13.577857 (+ 1966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:13.579504 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:13.579510 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:13.579512 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:13.579774 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:13.580040 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:13.580040 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:13.580041 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:13.580142 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:13.580337 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:13.580340 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:13.580456 (+ 116us) client_negotiation.cc:769] Sending connection context
0619 20:32:13.580536 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 20:32:13.580663 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":66,"threads_started":1}
I20260619 20:32:13.581420 12284 heartbeater.cc:347] Connected to a master server at 127.4.83.126:43281
I20260619 20:32:13.581502 12284 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:13.581671 12284 heartbeater.cc:511] Master 127.4.83.126:43281 requested a full tablet report, sending...
I20260619 20:32:13.582059 11939 ts_manager.cc:194] Registered new tserver with Master: 60525532c20a4c358974dc8e75b0596d (127.4.83.66:40003)
I20260619 20:32:13.582556 11939 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.83.66@KRBTEST.COM'} at 127.4.83.66:56743
I20260619 20:32:13.585543 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 12153
I20260619 20:32:13.585655 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance
I20260619 20:32:13.585880 4429 external_mini_cluster.cc:1428] Setting key d64d2a3f5405980b0a7bed8c0c0de4df
I20260619 20:32:13.587865 11990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:13.582459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:58539 (local address 127.4.83.126:43281)
0619 20:32:13.582496 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:13.582498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:13.582547 (+ 49us) server_negotiation.cc:407] Connection header received
0619 20:32:13.582612 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:13.582613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:13.582623 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:13.582679 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:13.583059 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.583541 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.584266 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.584468 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.584522 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:13.585414 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:13.585415 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:13.585415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:13.585422 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:13.586801 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:13.587096 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:13.587097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:13.587098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:13.587153 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:13.587313 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:13.587314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:13.587314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:13.587467 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:13.587525 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:13.587667 (+ 142us) server_negotiation.cc:299] Negotiation successful
0619 20:32:13.587745 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 20:32:13.587879 12290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:13.582377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:43281 (local address 127.4.83.66:58539)
0619 20:32:13.582499 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:13.582508 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:13.582559 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:13.582743 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:13.582744 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:13.582911 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:13.582963 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:13.582964 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.583629 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.583630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:13.584195 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:13.584197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.584561 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.584562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:13.584621 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:13.585054 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:13.585059 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:13.585332 (+ 273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:13.586867 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:13.586868 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:13.586869 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:13.587036 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:13.587205 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:13.587206 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:13.587206 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:13.587264 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:13.587528 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:13.587528 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:13.587577 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 20:32:13.587636 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 20:32:13.587747 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":62,"threads_started":1}
WARNING: no policy specified for oryx/127.4.83.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.83.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.83.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.67.keytab.
Entry for principal oryx/127.4.83.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.67.keytab.
I20260619 20:32:13.611344 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:0
--local_ip_for_outbound_sockets=127.4.83.67
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:43281
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.67.keytab
--principal=oryx/127.4.83.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:38389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:13.696875 12293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:13.697199 12293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:13.697228 12293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:13.698573 12293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:13.698626 12293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:13.698652 12293 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 20:32:13.698670 12293 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:32:13.700323 12293 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:38389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.67.keytab
--principal=oryx/127.4.83.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.83.126:43281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:13.700565 12293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:13.700799 12293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:13.706264 12301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:13.706363 12299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:13.706439 12293 server_base.cc:1034] running on GCE node
W20260619 20:32:13.706550 12298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:13.706740 12293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:13.706955 12293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:13.708130 12293 hybrid_clock.cc:648] HybridClock initialized: now 1781901133708112 us; error 34 us; skew 500 ppm
Jun 19 20:32:13 dist-test-slave-4m58 krb5kdc[11895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901133, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:13.711138 12293 init.cc:377] Logged in from keytab as oryx/127.4.83.67@KRBTEST.COM (short username oryx)
I20260619 20:32:13.711665 12293 webserver.cc:460] Webserver started at http://127.4.83.67:41535/ using document root <none> and password file <none>
I20260619 20:32:13.711952 12293 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:13.712008 12293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:13.712105 12293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:13.713251 12293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-2/data/instance:
uuid: "bcbfef3232264a4dbfa51874f4ac555e"
format_stamp: "Formatted at 2026-06-19 20:32:13 on dist-test-slave-4m58"
server_key: "2b7f88cc5670b992418542d03603d028"
server_key_iv: "8ef9030cb4944524a6c5b8c608b15a46"
server_key_version: "encryptionkey@0"
I20260619 20:32:13.713688 12293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance:
uuid: "bcbfef3232264a4dbfa51874f4ac555e"
format_stamp: "Formatted at 2026-06-19 20:32:13 on dist-test-slave-4m58"
server_key: "2b7f88cc5670b992418542d03603d028"
server_key_iv: "8ef9030cb4944524a6c5b8c608b15a46"
server_key_version: "encryptionkey@0"
I20260619 20:32:13.715147 12293 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:32:13.716187 12308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:13.716492 12293 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:32:13.716591 12293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "bcbfef3232264a4dbfa51874f4ac555e"
format_stamp: "Formatted at 2026-06-19 20:32:13 on dist-test-slave-4m58"
server_key: "2b7f88cc5670b992418542d03603d028"
server_key_iv: "8ef9030cb4944524a6c5b8c608b15a46"
server_key_version: "encryptionkey@0"
I20260619 20:32:13.716678 12293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:13.735291 12293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:13.737974 12293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:13.738125 12293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:13.738735 12293 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:13.738776 12293 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:13.738797 12293 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:13.738809 12293 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 20:32:13 dist-test-slave-4m58 krb5kdc[11895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901133, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM
I20260619 20:32:13.746184 12293 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:36719
I20260619 20:32:13.746361 12423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:36719 every 8 connection(s)
I20260619 20:32:13.746583 12293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
I20260619 20:32:13.746745 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 12293
I20260619 20:32:13.746843 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance
I20260619 20:32:13.747025 4429 external_mini_cluster.cc:1428] Setting key 0155a2e67c5a93b86baf68fa1c29fa02
I20260619 20:32:13.749876 11990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:13.739202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:56575 (local address 127.4.83.126:43281)
0619 20:32:13.739273 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:13.739276 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:13.739480 (+ 204us) server_negotiation.cc:407] Connection header received
0619 20:32:13.740226 (+ 746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:13.740228 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:13.740243 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:13.740303 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:13.741201 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.741725 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.743339 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.743641 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.743742 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:13.746786 (+ 3044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:13.746788 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:13.746789 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:13.746799 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:13.748424 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:13.748801 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:13.748802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:13.748803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:13.748878 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:13.749217 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:13.749218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:13.749219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:13.749344 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:13.749402 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:13.749692 (+ 290us) server_negotiation.cc:299] Negotiation successful
0619 20:32:13.749786 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 20:32:13.750062 12317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:13.739168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:43281 (local address 127.4.83.67:56575)
0619 20:32:13.739328 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:13.739376 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:13.740112 (+ 736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:13.740384 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:13.740389 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:13.740760 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:13.741083 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:13.741091 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.741821 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.741822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:13.743216 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:13.743219 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.743758 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.743759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:13.743958 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:13.744535 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:13.744546 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:13.746668 (+ 2122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:13.748501 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:13.748507 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:13.748509 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:13.748730 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:13.749031 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:13.749032 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:13.749033 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:13.749152 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:13.749403 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:13.749409 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:13.749551 (+ 142us) client_negotiation.cc:769] Sending connection context
0619 20:32:13.749656 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 20:32:13.749831 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260619 20:32:13.750746 12424 heartbeater.cc:347] Connected to a master server at 127.4.83.126:43281
I20260619 20:32:13.750833 12424 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:13.751013 12424 heartbeater.cc:511] Master 127.4.83.126:43281 requested a full tablet report, sending...
I20260619 20:32:13.751408 11940 ts_manager.cc:194] Registered new tserver with Master: bcbfef3232264a4dbfa51874f4ac555e (127.4.83.67:36719)
I20260619 20:32:13.751787 11940 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.83.67@KRBTEST.COM'} at 127.4.83.67:56575
I20260619 20:32:13.758077 11990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:13.751819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:57581 (local address 127.4.83.126:43281)
0619 20:32:13.751878 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:13.751880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:13.751944 (+ 64us) server_negotiation.cc:407] Connection header received
0619 20:32:13.752017 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:13.752018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:13.752028 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:13.752083 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:13.752384 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.752858 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.753515 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.753720 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.753773 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:13.754814 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:13.754815 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:13.754815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:13.754822 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:13.756289 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:13.756615 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:13.756617 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:13.756617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:13.756667 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:13.757176 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:13.757178 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:13.757179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:13.757509 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:13.757724 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:13.757906 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 20:32:13.757982 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 20:32:13.758100 12430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:13.751747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:43281 (local address 127.4.83.67:57581)
0619 20:32:13.751882 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:13.751892 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:13.751969 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:13.752088 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:13.752089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:13.752259 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:13.752319 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:13.752320 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.752943 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.752944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:13.753446 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:13.753448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.753779 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.753780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:13.753862 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:13.754411 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:13.754415 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:13.754731 (+ 316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:13.756384 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:13.756385 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:13.756385 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:13.756546 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:13.756840 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:13.756841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:13.756842 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:13.756906 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:13.757717 (+ 811us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:13.757718 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:13.757801 (+ 83us) client_negotiation.cc:769] Sending connection context
0619 20:32:13.757865 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 20:32:13.757978 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260619 20:32:13.758862 4429 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 20:32:13 dist-test-slave-4m58 krb5kdc[11895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901133, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 20:32:13 dist-test-slave-4m58 krb5kdc[11895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:13 dist-test-slave-4m58 krb5kdc[11895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
W20260619 20:32:13.783062 11990 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:13.775688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59898 (local address 127.4.83.126:43281)
0619 20:32:13.775776 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:13.775778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:13.775792 (+ 14us) server_negotiation.cc:407] Connection header received
0619 20:32:13.775822 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:13.775823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:13.775876 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:13.775952 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:13.776731 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.777263 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.778189 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.778393 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.778459 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:13.782944 (+ 4485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59898: BlockingRecv error: recv got EOF from 127.0.0.1:59898 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 19 20:32:13 dist-test-slave-4m58 krb5kdc[11895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901133, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM
I20260619 20:32:13.793453 11990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:13.784041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59900 (local address 127.4.83.126:43281)
0619 20:32:13.784089 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:13.784091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:13.784144 (+ 53us) server_negotiation.cc:407] Connection header received
0619 20:32:13.784218 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:13.784219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:13.784228 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:13.784280 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:13.784936 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.785482 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.786608 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.786829 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.786904 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:13.790422 (+ 3518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:13.790425 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:13.790425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:13.790436 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:13.791997 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:13.792476 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:13.792477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:13.792478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:13.792554 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:13.792802 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:13.792802 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:13.792803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:13.792946 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:13.793046 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:13.793266 (+ 220us) server_negotiation.cc:299] Negotiation successful
0619 20:32:13.793366 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:32:13.796092 11940 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:32:13.798692 11940 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:32:13.807934 12447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:13.802658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:40003 (local address 127.0.0.1:38276)
0619 20:32:13.802927 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:13.802992 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:13.803066 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:13.803532 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:13.803537 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:13.803540 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:13.803736 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:13.803740 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.805365 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.805367 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:13.806409 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:13.806413 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.807472 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.807473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:13.807567 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:13.807587 (+ 20us) client_negotiation.cc:769] Sending connection context
0619 20:32:13.807689 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 20:32:13.807769 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":30,"thread_start_us":93,"threads_started":1}
I20260619 20:32:13.807934 12446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:13.802688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:36719 (local address 127.0.0.1:53124)
0619 20:32:13.803020 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:13.803037 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:13.803120 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:13.803542 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:13.803543 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:13.803546 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:13.803712 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:13.803716 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.805093 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.805094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:13.806645 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:13.806647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.807632 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.807633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:13.807712 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:13.807716 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:32:13.807767 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 20:32:13.807802 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":35,"thread_start_us":151,"threads_started":1}
I20260619 20:32:13.808315 12449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:13.802835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38276 (local address 127.4.83.66:40003)
0619 20:32:13.803281 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:13.803288 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:13.803302 (+ 14us) server_negotiation.cc:407] Connection header received
0619 20:32:13.803362 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:13.803365 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:13.803386 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:13.803825 (+ 439us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:13.803971 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.805200 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.806510 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.807334 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.807449 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:13.807565 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:13.807724 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 20:32:13.807781 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":228,"threads_started":1}
I20260619 20:32:13.808365 12450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:13.803099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53124 (local address 127.4.83.67:36719)
0619 20:32:13.803268 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:13.803274 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:13.803289 (+ 15us) server_negotiation.cc:407] Connection header received
0619 20:32:13.803333 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:13.803335 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:13.803353 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:13.803436 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:13.804014 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.804983 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.806768 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.807530 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.807611 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:13.807676 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:13.808000 (+ 324us) server_negotiation.cc:299] Negotiation successful
0619 20:32:13.808261 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":103,"threads_started":1}
I20260619 20:32:13.809204 12358 tablet_service.cc:1467] Processing CreateTablet for tablet 0ed4ddce0c844bdd92f8030132d3df3a (DEFAULT_TABLE table=test-table [id=eb1c7c52781644f0863c069c6aa8d03d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:32:13.809291 12218 tablet_service.cc:1467] Processing CreateTablet for tablet 0ed4ddce0c844bdd92f8030132d3df3a (DEFAULT_TABLE table=test-table [id=eb1c7c52781644f0863c069c6aa8d03d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:32:13.809557 12448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:13.802915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:39085 (local address 127.0.0.1:48768)
0619 20:32:13.803204 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:13.803216 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:13.803273 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:13.803952 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:13.803953 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:13.803957 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:13.804016 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:13.804017 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.805322 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.805323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:13.806639 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:13.806643 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.809295 (+ 2652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.809296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:13.809359 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:13.809365 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 20:32:13.809442 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 20:32:13.809488 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":11,"thread_start_us":109,"threads_started":1}
I20260619 20:32:13.809639 12358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ed4ddce0c844bdd92f8030132d3df3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:13.809789 12218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ed4ddce0c844bdd92f8030132d3df3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:13.810077 12451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:13.803426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48768 (local address 127.4.83.65:39085)
0619 20:32:13.803697 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:13.803702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:13.803713 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:32:13.803758 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:13.803760 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:13.803776 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:13.803889 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:13.804108 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.805199 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.807731 (+ 2532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.809192 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.809280 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:13.809368 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:13.809766 (+ 398us) server_negotiation.cc:299] Negotiation successful
0619 20:32:13.809829 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":135,"threads_started":1}
I20260619 20:32:13.811129 12078 tablet_service.cc:1467] Processing CreateTablet for tablet 0ed4ddce0c844bdd92f8030132d3df3a (DEFAULT_TABLE table=test-table [id=eb1c7c52781644f0863c069c6aa8d03d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:32:13.811548 12078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ed4ddce0c844bdd92f8030132d3df3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:13.813601 12452 tablet_bootstrap.cc:492] T 0ed4ddce0c844bdd92f8030132d3df3a P bcbfef3232264a4dbfa51874f4ac555e: Bootstrap starting.
I20260619 20:32:13.814424 12453 tablet_bootstrap.cc:492] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d: Bootstrap starting.
I20260619 20:32:13.814795 12452 tablet_bootstrap.cc:654] T 0ed4ddce0c844bdd92f8030132d3df3a P bcbfef3232264a4dbfa51874f4ac555e: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:13.815120 12453 tablet_bootstrap.cc:654] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:13.815487 12453 log.cc:824] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:13.815475 12452 log.cc:824] T 0ed4ddce0c844bdd92f8030132d3df3a P bcbfef3232264a4dbfa51874f4ac555e: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:13.815598 12454 tablet_bootstrap.cc:492] T 0ed4ddce0c844bdd92f8030132d3df3a P 9db5cca45bf244168b2aa2cfd5454764: Bootstrap starting.
I20260619 20:32:13.816397 12453 tablet_bootstrap.cc:492] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d: No bootstrap required, opened a new log
I20260619 20:32:13.816442 12454 tablet_bootstrap.cc:654] T 0ed4ddce0c844bdd92f8030132d3df3a P 9db5cca45bf244168b2aa2cfd5454764: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:13.816473 12453 ts_tablet_manager.cc:1397] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:32:13.816720 12452 tablet_bootstrap.cc:492] T 0ed4ddce0c844bdd92f8030132d3df3a P bcbfef3232264a4dbfa51874f4ac555e: No bootstrap required, opened a new log
I20260619 20:32:13.816892 12452 ts_tablet_manager.cc:1397] T 0ed4ddce0c844bdd92f8030132d3df3a P bcbfef3232264a4dbfa51874f4ac555e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 20:32:13.816890 12454 log.cc:824] T 0ed4ddce0c844bdd92f8030132d3df3a P 9db5cca45bf244168b2aa2cfd5454764: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:13.817998 12454 tablet_bootstrap.cc:492] T 0ed4ddce0c844bdd92f8030132d3df3a P 9db5cca45bf244168b2aa2cfd5454764: No bootstrap required, opened a new log
I20260619 20:32:13.818089 12454 ts_tablet_manager.cc:1397] T 0ed4ddce0c844bdd92f8030132d3df3a P 9db5cca45bf244168b2aa2cfd5454764: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 20:32:13.818425 12453 raft_consensus.cc:348] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } }
I20260619 20:32:13.818591 12453 raft_consensus.cc:374] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:13.818624 12453 raft_consensus.cc:729] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60525532c20a4c358974dc8e75b0596d, State: Initialized, Role: FOLLOWER
I20260619 20:32:13.818763 12453 consensus_queue.cc:260] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } }
I20260619 20:32:13.819088 12284 heartbeater.cc:503] Master 127.4.83.126:43281 was elected leader, sending a full tablet report...
I20260619 20:32:13.819087 12453 ts_tablet_manager.cc:1428] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:32:13.819087 12452 raft_consensus.cc:348] T 0ed4ddce0c844bdd92f8030132d3df3a P bcbfef3232264a4dbfa51874f4ac555e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } }
I20260619 20:32:13.819279 12452 raft_consensus.cc:374] T 0ed4ddce0c844bdd92f8030132d3df3a P bcbfef3232264a4dbfa51874f4ac555e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:13.819316 12452 raft_consensus.cc:729] T 0ed4ddce0c844bdd92f8030132d3df3a P bcbfef3232264a4dbfa51874f4ac555e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcbfef3232264a4dbfa51874f4ac555e, State: Initialized, Role: FOLLOWER
I20260619 20:32:13.819443 12452 consensus_queue.cc:260] T 0ed4ddce0c844bdd92f8030132d3df3a P bcbfef3232264a4dbfa51874f4ac555e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } }
I20260619 20:32:13.819782 12452 ts_tablet_manager.cc:1428] T 0ed4ddce0c844bdd92f8030132d3df3a P bcbfef3232264a4dbfa51874f4ac555e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:32:13.819950 12424 heartbeater.cc:503] Master 127.4.83.126:43281 was elected leader, sending a full tablet report...
I20260619 20:32:13.820502 12454 raft_consensus.cc:348] T 0ed4ddce0c844bdd92f8030132d3df3a P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } }
I20260619 20:32:13.820677 12454 raft_consensus.cc:374] T 0ed4ddce0c844bdd92f8030132d3df3a P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:13.820713 12454 raft_consensus.cc:729] T 0ed4ddce0c844bdd92f8030132d3df3a P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9db5cca45bf244168b2aa2cfd5454764, State: Initialized, Role: FOLLOWER
I20260619 20:32:13.820875 12454 consensus_queue.cc:260] T 0ed4ddce0c844bdd92f8030132d3df3a P 9db5cca45bf244168b2aa2cfd5454764 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } }
I20260619 20:32:13.821166 12454 ts_tablet_manager.cc:1428] T 0ed4ddce0c844bdd92f8030132d3df3a P 9db5cca45bf244168b2aa2cfd5454764: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:32:13.821220 12144 heartbeater.cc:503] Master 127.4.83.126:43281 was elected leader, sending a full tablet report...
W20260619 20:32:13.829283 12285 tablet.cc:2369] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:32:13.835603 12458 raft_consensus.cc:482] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:32:13.835779 12458 raft_consensus.cc:504] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } }
I20260619 20:32:13.836241 12458 leader_election.cc:288] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcbfef3232264a4dbfa51874f4ac555e (127.4.83.67:36719), 9db5cca45bf244168b2aa2cfd5454764 (127.4.83.65:39085)
I20260619 20:32:13.840052 12177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:13.836500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:36719 (local address 127.4.83.66:33071)
0619 20:32:13.836569 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:13.836583 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:13.836636 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:13.836822 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:13.836823 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:13.836826 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:13.836916 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:13.836918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.837770 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.837771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:13.838783 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:13.838787 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.839600 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.839601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:13.839699 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:13.839705 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 20:32:13.839801 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 20:32:13.839948 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 20:32:13.840055 12450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:13.836549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:33071 (local address 127.4.83.67:36719)
0619 20:32:13.836639 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:13.836641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:13.836689 (+ 48us) server_negotiation.cc:407] Connection header received
0619 20:32:13.836721 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:13.836722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:13.836732 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:13.836785 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:13.837034 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.837656 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.838944 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.839500 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.839553 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:13.839565 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:13.839814 (+ 249us) server_negotiation.cc:299] Negotiation successful
0619 20:32:13.839961 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 20:32:13.840623 12378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ed4ddce0c844bdd92f8030132d3df3a" candidate_uuid: "60525532c20a4c358974dc8e75b0596d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcbfef3232264a4dbfa51874f4ac555e" is_pre_election: true
I20260619 20:32:13.840807 12461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:13.836511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:39085 (local address 127.4.83.66:37363)
0619 20:32:13.836984 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:13.837001 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:13.837093 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:13.837240 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:13.837241 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:13.837243 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:13.837345 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:13.837347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.838258 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.838259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:13.839351 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:13.839356 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.840480 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.840481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:13.840567 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:13.840574 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 20:32:13.840647 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 20:32:13.840699 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":166,"thread_start_us":152,"threads_started":1}
I20260619 20:32:13.840814 12378 raft_consensus.cc:2454] T 0ed4ddce0c844bdd92f8030132d3df3a P bcbfef3232264a4dbfa51874f4ac555e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60525532c20a4c358974dc8e75b0596d in term 0.
I20260619 20:32:13.840811 12451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:13.836583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:37363 (local address 127.4.83.65:39085)
0619 20:32:13.836666 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:13.836668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:13.837089 (+ 421us) server_negotiation.cc:407] Connection header received
0619 20:32:13.837160 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:13.837161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:13.837169 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:13.837215 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:13.837453 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.838126 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.839481 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.840191 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.840460 (+ 269us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:13.840475 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:13.840691 (+ 216us) server_negotiation.cc:299] Negotiation successful
0619 20:32:13.840736 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 20:32:13.841197 12171 leader_election.cc:302] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60525532c20a4c358974dc8e75b0596d, bcbfef3232264a4dbfa51874f4ac555e; no voters:
I20260619 20:32:13.841248 12098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ed4ddce0c844bdd92f8030132d3df3a" candidate_uuid: "60525532c20a4c358974dc8e75b0596d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9db5cca45bf244168b2aa2cfd5454764" is_pre_election: true
I20260619 20:32:13.841387 12098 raft_consensus.cc:2454] T 0ed4ddce0c844bdd92f8030132d3df3a P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60525532c20a4c358974dc8e75b0596d in term 0.
I20260619 20:32:13.841413 12458 raft_consensus.cc:2781] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:32:13.841513 12458 raft_consensus.cc:482] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:32:13.841548 12458 raft_consensus.cc:3037] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:13.842720 12458 raft_consensus.cc:504] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } }
I20260619 20:32:13.842902 12458 leader_election.cc:288] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d [CANDIDATE]: Term 1 election: Requested vote from peers bcbfef3232264a4dbfa51874f4ac555e (127.4.83.67:36719), 9db5cca45bf244168b2aa2cfd5454764 (127.4.83.65:39085)
I20260619 20:32:13.843102 12098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ed4ddce0c844bdd92f8030132d3df3a" candidate_uuid: "60525532c20a4c358974dc8e75b0596d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9db5cca45bf244168b2aa2cfd5454764"
I20260619 20:32:13.843148 12378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ed4ddce0c844bdd92f8030132d3df3a" candidate_uuid: "60525532c20a4c358974dc8e75b0596d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcbfef3232264a4dbfa51874f4ac555e"
I20260619 20:32:13.843216 12378 raft_consensus.cc:3037] T 0ed4ddce0c844bdd92f8030132d3df3a P bcbfef3232264a4dbfa51874f4ac555e [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:13.843216 12098 raft_consensus.cc:3037] T 0ed4ddce0c844bdd92f8030132d3df3a P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:13.844231 12098 raft_consensus.cc:2454] T 0ed4ddce0c844bdd92f8030132d3df3a P 9db5cca45bf244168b2aa2cfd5454764 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60525532c20a4c358974dc8e75b0596d in term 1.
I20260619 20:32:13.844231 12378 raft_consensus.cc:2454] T 0ed4ddce0c844bdd92f8030132d3df3a P bcbfef3232264a4dbfa51874f4ac555e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60525532c20a4c358974dc8e75b0596d in term 1.
I20260619 20:32:13.844475 12171 leader_election.cc:302] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60525532c20a4c358974dc8e75b0596d, bcbfef3232264a4dbfa51874f4ac555e; no voters:
I20260619 20:32:13.844604 12458 raft_consensus.cc:2781] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:32:13.844807 12458 raft_consensus.cc:686] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d [term 1 LEADER]: Becoming Leader. State: Replica: 60525532c20a4c358974dc8e75b0596d, State: Running, Role: LEADER
I20260619 20:32:13.844951 12458 consensus_queue.cc:237] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } }
I20260619 20:32:13.846035 11939 catalog_manager.cc:5495] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d reported cstate change: term changed from 0 to 1, leader changed from <none> to 60525532c20a4c358974dc8e75b0596d (127.4.83.66). New cstate: current_term: 1 leader_uuid: "60525532c20a4c358974dc8e75b0596d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } health_report { overall_health: UNKNOWN } } }
I20260619 20:32:13.855962 12449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:13.853069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38278 (local address 127.4.83.66:40003)
0619 20:32:13.853147 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:13.853150 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:13.853158 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:32:13.853202 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:13.853203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:13.853229 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:13.853308 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:32:13.853540 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.854073 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.854981 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:13.855205 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:13.855281 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:13.855514 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:32:13.855660 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:32:13.855746 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:13.855804 (+ 58us) server_negotiation.cc:299] Negotiation successful
0619 20:32:13.855859 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 20:32:13.858419 12098 raft_consensus.cc:1261] T 0ed4ddce0c844bdd92f8030132d3df3a P 9db5cca45bf244168b2aa2cfd5454764 [term 1 FOLLOWER]: Refusing update from remote peer 60525532c20a4c358974dc8e75b0596d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:32:13.858412 12378 raft_consensus.cc:1261] T 0ed4ddce0c844bdd92f8030132d3df3a P bcbfef3232264a4dbfa51874f4ac555e [term 1 FOLLOWER]: Refusing update from remote peer 60525532c20a4c358974dc8e75b0596d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:32:13.858850 12462 consensus_queue.cc:1035] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:32:13.859028 12458 consensus_queue.cc:1035] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:32:13.862473 12468 mvcc.cc:205] Tried to move back new op lower bound from 7298667044280786944 to 7298667044229709824. Current Snapshot: MvccSnapshot[applied={T|T < 7298667044280786944}]
I20260619 20:32:13.862926 12466 mvcc.cc:205] Tried to move back new op lower bound from 7298667044280786944 to 7298667044229709824. Current Snapshot: MvccSnapshot[applied={T|T < 7298667044280786944 or (T in {7298667044280786944})}]
I20260619 20:32:13.864122 11939 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:32:13.864238 11939 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:32:13.865114 11939 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 74439e8536d84e1884988644eba75b84: Sending DeleteTablet for 3 replicas of tablet 0ed4ddce0c844bdd92f8030132d3df3a
I20260619 20:32:13.865475 12358 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ed4ddce0c844bdd92f8030132d3df3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:32:13 UTC) from {username='oryx', principal='oryx/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:53124
I20260619 20:32:13.865502 12078 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ed4ddce0c844bdd92f8030132d3df3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:32:13 UTC) from {username='oryx', principal='oryx/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:48768
I20260619 20:32:13.865473 12218 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ed4ddce0c844bdd92f8030132d3df3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:32:13 UTC) from {username='oryx', principal='oryx/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:38276
I20260619 20:32:13.866035 12474 tablet_replica.cc:330] T 0ed4ddce0c844bdd92f8030132d3df3a P 9db5cca45bf244168b2aa2cfd5454764: stopping tablet replica
I20260619 20:32:13.866065 11939 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 20:32:13.866159 12474 raft_consensus.cc:2229] T 0ed4ddce0c844bdd92f8030132d3df3a P 9db5cca45bf244168b2aa2cfd5454764 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260619 20:32:13.866227 11939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:32:13.866282 12474 raft_consensus.cc:2258] T 0ed4ddce0c844bdd92f8030132d3df3a P 9db5cca45bf244168b2aa2cfd5454764 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:32:13.866544 12475 tablet_replica.cc:330] T 0ed4ddce0c844bdd92f8030132d3df3a P bcbfef3232264a4dbfa51874f4ac555e: stopping tablet replica
I20260619 20:32:13.866806 12474 ts_tablet_manager.cc:1905] T 0ed4ddce0c844bdd92f8030132d3df3a P 9db5cca45bf244168b2aa2cfd5454764: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:32:13.868014 12078 tablet_service.cc:1467] Processing CreateTablet for tablet 676c95bdc427474dadbb7ed9fabc13a9 (DEFAULT_TABLE table=test-table [id=4b84a09d99c74dd4812e2ef610a5412a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:32:13.867995 12218 tablet_service.cc:1467] Processing CreateTablet for tablet 676c95bdc427474dadbb7ed9fabc13a9 (DEFAULT_TABLE table=test-table [id=4b84a09d99c74dd4812e2ef610a5412a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:32:13.868131 12078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 676c95bdc427474dadbb7ed9fabc13a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:13.868180 12218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 676c95bdc427474dadbb7ed9fabc13a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:13.868662 12474 ts_tablet_manager.cc:1918] T 0ed4ddce0c844bdd92f8030132d3df3a P 9db5cca45bf244168b2aa2cfd5454764: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:32:13.868729 12474 log.cc:1196] T 0ed4ddce0c844bdd92f8030132d3df3a P 9db5cca45bf244168b2aa2cfd5454764: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-0/wal/wals/0ed4ddce0c844bdd92f8030132d3df3a
I20260619 20:32:13.868984 12474 ts_tablet_manager.cc:1939] T 0ed4ddce0c844bdd92f8030132d3df3a P 9db5cca45bf244168b2aa2cfd5454764: Deleting consensus metadata
I20260619 20:32:13.869379 12454 tablet_bootstrap.cc:492] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764: Bootstrap starting.
I20260619 20:32:13.869738 12358 tablet_service.cc:1467] Processing CreateTablet for tablet 676c95bdc427474dadbb7ed9fabc13a9 (DEFAULT_TABLE table=test-table [id=4b84a09d99c74dd4812e2ef610a5412a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:32:13.869786 12454 tablet_bootstrap.cc:654] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:13.869855 12358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 676c95bdc427474dadbb7ed9fabc13a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:13.869890 12453 tablet_bootstrap.cc:492] T 676c95bdc427474dadbb7ed9fabc13a9 P 60525532c20a4c358974dc8e75b0596d: Bootstrap starting.
I20260619 20:32:13.870215 11927 catalog_manager.cc:4841] TS 9db5cca45bf244168b2aa2cfd5454764 (127.4.83.65:39085): tablet 0ed4ddce0c844bdd92f8030132d3df3a (table test-table [id=eb1c7c52781644f0863c069c6aa8d03d]) successfully deleted
I20260619 20:32:13.870388 12453 tablet_bootstrap.cc:654] T 676c95bdc427474dadbb7ed9fabc13a9 P 60525532c20a4c358974dc8e75b0596d: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:13.870745 12454 tablet_bootstrap.cc:492] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764: No bootstrap required, opened a new log
I20260619 20:32:13.870806 12454 ts_tablet_manager.cc:1397] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:13.870956 12454 raft_consensus.cc:348] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } }
I20260619 20:32:13.871059 12454 raft_consensus.cc:374] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:13.871090 12454 raft_consensus.cc:729] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9db5cca45bf244168b2aa2cfd5454764, State: Initialized, Role: FOLLOWER
I20260619 20:32:13.871135 12454 consensus_queue.cc:260] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } }
I20260619 20:32:13.871234 12454 ts_tablet_manager.cc:1428] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 20:32:13.871343 12452 tablet_bootstrap.cc:492] T 676c95bdc427474dadbb7ed9fabc13a9 P bcbfef3232264a4dbfa51874f4ac555e: Bootstrap starting.
I20260619 20:32:13.871678 12453 tablet_bootstrap.cc:492] T 676c95bdc427474dadbb7ed9fabc13a9 P 60525532c20a4c358974dc8e75b0596d: No bootstrap required, opened a new log
I20260619 20:32:13.871747 12453 ts_tablet_manager.cc:1397] T 676c95bdc427474dadbb7ed9fabc13a9 P 60525532c20a4c358974dc8e75b0596d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:32:13.871762 12452 tablet_bootstrap.cc:654] T 676c95bdc427474dadbb7ed9fabc13a9 P bcbfef3232264a4dbfa51874f4ac555e: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:13.871985 12453 raft_consensus.cc:348] T 676c95bdc427474dadbb7ed9fabc13a9 P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } }
I20260619 20:32:13.872076 12453 raft_consensus.cc:374] T 676c95bdc427474dadbb7ed9fabc13a9 P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:13.872097 12453 raft_consensus.cc:729] T 676c95bdc427474dadbb7ed9fabc13a9 P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60525532c20a4c358974dc8e75b0596d, State: Initialized, Role: FOLLOWER
I20260619 20:32:13.872148 12453 consensus_queue.cc:260] T 676c95bdc427474dadbb7ed9fabc13a9 P 60525532c20a4c358974dc8e75b0596d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } }
I20260619 20:32:13.872251 12453 ts_tablet_manager.cc:1428] T 676c95bdc427474dadbb7ed9fabc13a9 P 60525532c20a4c358974dc8e75b0596d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:13.872438 12475 raft_consensus.cc:2229] T 0ed4ddce0c844bdd92f8030132d3df3a P bcbfef3232264a4dbfa51874f4ac555e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:32:13.872601 12475 raft_consensus.cc:2258] T 0ed4ddce0c844bdd92f8030132d3df3a P bcbfef3232264a4dbfa51874f4ac555e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:32:13.873132 12475 ts_tablet_manager.cc:1905] T 0ed4ddce0c844bdd92f8030132d3df3a P bcbfef3232264a4dbfa51874f4ac555e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:32:13.874722 12475 ts_tablet_manager.cc:1918] T 0ed4ddce0c844bdd92f8030132d3df3a P bcbfef3232264a4dbfa51874f4ac555e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:32:13.874788 12475 log.cc:1196] T 0ed4ddce0c844bdd92f8030132d3df3a P bcbfef3232264a4dbfa51874f4ac555e: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-2/wal/wals/0ed4ddce0c844bdd92f8030132d3df3a
I20260619 20:32:13.875003 12475 ts_tablet_manager.cc:1939] T 0ed4ddce0c844bdd92f8030132d3df3a P bcbfef3232264a4dbfa51874f4ac555e: Deleting consensus metadata
I20260619 20:32:13.875757 11926 catalog_manager.cc:4841] TS bcbfef3232264a4dbfa51874f4ac555e (127.4.83.67:36719): tablet 0ed4ddce0c844bdd92f8030132d3df3a (table test-table [id=eb1c7c52781644f0863c069c6aa8d03d]) successfully deleted
I20260619 20:32:13.879944 12476 tablet_replica.cc:330] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d: stopping tablet replica
I20260619 20:32:13.880038 12452 tablet_bootstrap.cc:492] T 676c95bdc427474dadbb7ed9fabc13a9 P bcbfef3232264a4dbfa51874f4ac555e: No bootstrap required, opened a new log
I20260619 20:32:13.880096 12452 ts_tablet_manager.cc:1397] T 676c95bdc427474dadbb7ed9fabc13a9 P bcbfef3232264a4dbfa51874f4ac555e: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20260619 20:32:13.880159 12476 raft_consensus.cc:2229] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:32:13.880267 12452 raft_consensus.cc:348] T 676c95bdc427474dadbb7ed9fabc13a9 P bcbfef3232264a4dbfa51874f4ac555e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } }
I20260619 20:32:13.880324 12476 raft_consensus.cc:2258] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:32:13.880348 12452 raft_consensus.cc:374] T 676c95bdc427474dadbb7ed9fabc13a9 P bcbfef3232264a4dbfa51874f4ac555e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:13.880371 12452 raft_consensus.cc:729] T 676c95bdc427474dadbb7ed9fabc13a9 P bcbfef3232264a4dbfa51874f4ac555e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcbfef3232264a4dbfa51874f4ac555e, State: Initialized, Role: FOLLOWER
I20260619 20:32:13.880421 12452 consensus_queue.cc:260] T 676c95bdc427474dadbb7ed9fabc13a9 P bcbfef3232264a4dbfa51874f4ac555e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } }
I20260619 20:32:13.880517 12452 ts_tablet_manager.cc:1428] T 676c95bdc427474dadbb7ed9fabc13a9 P bcbfef3232264a4dbfa51874f4ac555e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 20:32:13.881014 12476 ts_tablet_manager.cc:1905] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:32:13.882992 12476 ts_tablet_manager.cc:1918] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:32:13.883081 12476 log.cc:1196] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-1/wal/wals/0ed4ddce0c844bdd92f8030132d3df3a
I20260619 20:32:13.883337 12476 ts_tablet_manager.cc:1939] T 0ed4ddce0c844bdd92f8030132d3df3a P 60525532c20a4c358974dc8e75b0596d: Deleting consensus metadata
I20260619 20:32:13.883767 11927 catalog_manager.cc:4841] TS 60525532c20a4c358974dc8e75b0596d (127.4.83.66:40003): tablet 0ed4ddce0c844bdd92f8030132d3df3a (table test-table [id=eb1c7c52781644f0863c069c6aa8d03d]) successfully deleted
W20260619 20:32:13.918967 12145 tablet.cc:2369] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:32:13.997190 12425 tablet.cc:2369] T 676c95bdc427474dadbb7ed9fabc13a9 P bcbfef3232264a4dbfa51874f4ac555e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:32:14.080225 12460 raft_consensus.cc:482] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:32:14.080484 12460 raft_consensus.cc:504] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } }
I20260619 20:32:14.081063 12460 leader_election.cc:288] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60525532c20a4c358974dc8e75b0596d (127.4.83.66:40003), bcbfef3232264a4dbfa51874f4ac555e (127.4.83.67:36719)
I20260619 20:32:14.085953 12449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:14.081361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:43617 (local address 127.4.83.66:40003)
0619 20:32:14.081439 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:14.081442 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:14.081520 (+ 78us) server_negotiation.cc:407] Connection header received
0619 20:32:14.081618 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:14.081619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:14.081631 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:14.081747 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:14.082013 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.082854 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.084625 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.085410 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.085531 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:14.085547 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:14.085782 (+ 235us) server_negotiation.cc:299] Negotiation successful
0619 20:32:14.085840 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 20:32:14.085942 12450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:14.081370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:57589 (local address 127.4.83.67:36719)
0619 20:32:14.081438 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:14.081440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:14.081536 (+ 96us) server_negotiation.cc:407] Connection header received
0619 20:32:14.081617 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:14.081618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:14.081631 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:14.081739 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:14.082013 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.082842 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.084629 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.085420 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.085532 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:14.085547 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:14.085799 (+ 252us) server_negotiation.cc:299] Negotiation successful
0619 20:32:14.085850 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 20:32:14.085984 12481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:14.081244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:36719 (local address 127.4.83.65:57589)
0619 20:32:14.081419 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:14.081432 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:14.081535 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:14.081763 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:14.081764 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:14.081769 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:14.081889 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:14.081892 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.082984 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.082985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:14.084464 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:14.084465 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.085593 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.085593 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:14.085670 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:14.085679 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 20:32:14.085763 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 20:32:14.085826 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":96,"thread_start_us":67,"threads_started":1}
I20260619 20:32:14.085983 12480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:14.081244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:40003 (local address 127.4.83.65:43617)
0619 20:32:14.081420 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:14.081432 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:14.081535 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:14.081763 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:14.081764 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:14.081767 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:14.081889 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:14.081892 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.082992 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.082993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:14.084462 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:14.084465 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.085563 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.085565 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:14.085670 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:14.085676 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 20:32:14.085763 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 20:32:14.085826 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":128,"thread_start_us":71,"threads_started":1}
I20260619 20:32:14.086432 12378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "676c95bdc427474dadbb7ed9fabc13a9" candidate_uuid: "9db5cca45bf244168b2aa2cfd5454764" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcbfef3232264a4dbfa51874f4ac555e" is_pre_election: true
I20260619 20:32:14.086567 12378 raft_consensus.cc:2454] T 676c95bdc427474dadbb7ed9fabc13a9 P bcbfef3232264a4dbfa51874f4ac555e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9db5cca45bf244168b2aa2cfd5454764 in term 0.
I20260619 20:32:14.086597 12236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "676c95bdc427474dadbb7ed9fabc13a9" candidate_uuid: "9db5cca45bf244168b2aa2cfd5454764" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60525532c20a4c358974dc8e75b0596d" is_pre_election: true
I20260619 20:32:14.086760 12236 raft_consensus.cc:2454] T 676c95bdc427474dadbb7ed9fabc13a9 P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9db5cca45bf244168b2aa2cfd5454764 in term 0.
I20260619 20:32:14.086817 12030 leader_election.cc:302] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9db5cca45bf244168b2aa2cfd5454764, bcbfef3232264a4dbfa51874f4ac555e; no voters:
I20260619 20:32:14.087036 12460 raft_consensus.cc:2781] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:32:14.087096 12460 raft_consensus.cc:482] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:32:14.087111 12460 raft_consensus.cc:3037] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:14.087803 12460 raft_consensus.cc:504] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } }
I20260619 20:32:14.088006 12460 leader_election.cc:288] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764 [CANDIDATE]: Term 1 election: Requested vote from peers 60525532c20a4c358974dc8e75b0596d (127.4.83.66:40003), bcbfef3232264a4dbfa51874f4ac555e (127.4.83.67:36719)
I20260619 20:32:14.088291 12236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "676c95bdc427474dadbb7ed9fabc13a9" candidate_uuid: "9db5cca45bf244168b2aa2cfd5454764" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60525532c20a4c358974dc8e75b0596d"
I20260619 20:32:14.088390 12236 raft_consensus.cc:3037] T 676c95bdc427474dadbb7ed9fabc13a9 P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:14.088332 12378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "676c95bdc427474dadbb7ed9fabc13a9" candidate_uuid: "9db5cca45bf244168b2aa2cfd5454764" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcbfef3232264a4dbfa51874f4ac555e"
I20260619 20:32:14.088444 12378 raft_consensus.cc:3037] T 676c95bdc427474dadbb7ed9fabc13a9 P bcbfef3232264a4dbfa51874f4ac555e [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:14.089049 12378 raft_consensus.cc:2454] T 676c95bdc427474dadbb7ed9fabc13a9 P bcbfef3232264a4dbfa51874f4ac555e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9db5cca45bf244168b2aa2cfd5454764 in term 1.
I20260619 20:32:14.089341 12030 leader_election.cc:302] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9db5cca45bf244168b2aa2cfd5454764, bcbfef3232264a4dbfa51874f4ac555e; no voters:
I20260619 20:32:14.089390 12236 raft_consensus.cc:2454] T 676c95bdc427474dadbb7ed9fabc13a9 P 60525532c20a4c358974dc8e75b0596d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9db5cca45bf244168b2aa2cfd5454764 in term 1.
I20260619 20:32:14.089454 12460 raft_consensus.cc:2781] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:32:14.089573 12460 raft_consensus.cc:686] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764 [term 1 LEADER]: Becoming Leader. State: Replica: 9db5cca45bf244168b2aa2cfd5454764, State: Running, Role: LEADER
I20260619 20:32:14.089663 12460 consensus_queue.cc:237] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } }
I20260619 20:32:14.090581 11939 catalog_manager.cc:5495] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9db5cca45bf244168b2aa2cfd5454764 (127.4.83.65). New cstate: current_term: 1 leader_uuid: "9db5cca45bf244168b2aa2cfd5454764" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } health_report { overall_health: UNKNOWN } } }
Jun 19 20:32:14 dist-test-slave-4m58 krb5kdc[11895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901133, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.126@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM
I20260619 20:32:14.136147 12489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:14.128816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:43281 (local address 127.0.0.1:59914)
0619 20:32:14.129046 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:14.129059 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:14.129182 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:14.129322 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:14.129323 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:14.129675 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:14.129725 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:14.129727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.130584 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.130585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:14.131125 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:14.131127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.131438 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.131439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:14.131486 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:14.132091 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:14.132096 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:14.133422 (+ 1326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:14.135121 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:14.135130 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:14.135132 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:14.135308 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:14.135474 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:14.135475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:14.135475 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:14.135526 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:14.135733 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:14.135741 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:14.135785 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 20:32:14.135866 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 20:32:14.136027 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":1,"thread_start_us":127,"threads_started":1}
I20260619 20:32:14.136186 11990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:14.128893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59914 (local address 127.4.83.126:43281)
0619 20:32:14.128990 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:14.128992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:14.129171 (+ 179us) server_negotiation.cc:407] Connection header received
0619 20:32:14.129259 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:14.129260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:14.129270 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:14.129312 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:14.129825 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.130442 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.131204 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.131380 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.131433 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:14.133504 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:14.133505 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:14.133506 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:14.133512 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:14.135038 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:14.135372 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:14.135373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:14.135373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:14.135423 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:14.135574 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:14.135574 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:14.135575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:14.135683 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:14.135736 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:14.135890 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 20:32:14.136086 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":120,"server-negotiator.queue_time_us":47}
I20260619 20:32:14.139106 11939 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:59914:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 20:32:14.139406 11939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:32:14.141386 12358 tablet_service.cc:1467] Processing CreateTablet for tablet ee7b48a88ff84bf2beeb421986bb558b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0207723976ec4ed6984c35e175d90095]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 20:32:14.141392 12078 tablet_service.cc:1467] Processing CreateTablet for tablet ee7b48a88ff84bf2beeb421986bb558b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0207723976ec4ed6984c35e175d90095]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 20:32:14.141456 12218 tablet_service.cc:1467] Processing CreateTablet for tablet ee7b48a88ff84bf2beeb421986bb558b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0207723976ec4ed6984c35e175d90095]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 20:32:14.141571 12358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee7b48a88ff84bf2beeb421986bb558b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:14.141571 12078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee7b48a88ff84bf2beeb421986bb558b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:14.141708 12218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee7b48a88ff84bf2beeb421986bb558b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:14.143198 12454 tablet_bootstrap.cc:492] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764: Bootstrap starting.
I20260619 20:32:14.143198 12452 tablet_bootstrap.cc:492] T ee7b48a88ff84bf2beeb421986bb558b P bcbfef3232264a4dbfa51874f4ac555e: Bootstrap starting.
I20260619 20:32:14.143322 12453 tablet_bootstrap.cc:492] T ee7b48a88ff84bf2beeb421986bb558b P 60525532c20a4c358974dc8e75b0596d: Bootstrap starting.
I20260619 20:32:14.143639 12454 tablet_bootstrap.cc:654] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:14.143639 12452 tablet_bootstrap.cc:654] T ee7b48a88ff84bf2beeb421986bb558b P bcbfef3232264a4dbfa51874f4ac555e: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:14.143919 12453 tablet_bootstrap.cc:654] T ee7b48a88ff84bf2beeb421986bb558b P 60525532c20a4c358974dc8e75b0596d: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:14.144341 12454 tablet_bootstrap.cc:492] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764: No bootstrap required, opened a new log
I20260619 20:32:14.144385 12454 ts_tablet_manager.cc:1397] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 20:32:14.144528 12454 raft_consensus.cc:348] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } }
I20260619 20:32:14.144613 12454 raft_consensus.cc:374] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:14.144620 12452 tablet_bootstrap.cc:492] T ee7b48a88ff84bf2beeb421986bb558b P bcbfef3232264a4dbfa51874f4ac555e: No bootstrap required, opened a new log
I20260619 20:32:14.144632 12454 raft_consensus.cc:729] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9db5cca45bf244168b2aa2cfd5454764, State: Initialized, Role: FOLLOWER
I20260619 20:32:14.144657 12452 ts_tablet_manager.cc:1397] T ee7b48a88ff84bf2beeb421986bb558b P bcbfef3232264a4dbfa51874f4ac555e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 20:32:14.144685 12454 consensus_queue.cc:260] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } }
I20260619 20:32:14.144781 12454 ts_tablet_manager.cc:1428] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:14.144781 12452 raft_consensus.cc:348] T ee7b48a88ff84bf2beeb421986bb558b P bcbfef3232264a4dbfa51874f4ac555e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } }
I20260619 20:32:14.144861 12452 raft_consensus.cc:374] T ee7b48a88ff84bf2beeb421986bb558b P bcbfef3232264a4dbfa51874f4ac555e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:14.144881 12452 raft_consensus.cc:729] T ee7b48a88ff84bf2beeb421986bb558b P bcbfef3232264a4dbfa51874f4ac555e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcbfef3232264a4dbfa51874f4ac555e, State: Initialized, Role: FOLLOWER
I20260619 20:32:14.144914 12452 consensus_queue.cc:260] T ee7b48a88ff84bf2beeb421986bb558b P bcbfef3232264a4dbfa51874f4ac555e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } }
I20260619 20:32:14.144994 12452 ts_tablet_manager.cc:1428] T ee7b48a88ff84bf2beeb421986bb558b P bcbfef3232264a4dbfa51874f4ac555e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:14.144896 12460 tablet_replica.cc:439] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } }
I20260619 20:32:14.145033 12460 tablet_replica.cc:442] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:32:14.145207 12459 tablet_replica.cc:439] T ee7b48a88ff84bf2beeb421986bb558b P bcbfef3232264a4dbfa51874f4ac555e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } }
I20260619 20:32:14.145346 12459 tablet_replica.cc:442] T ee7b48a88ff84bf2beeb421986bb558b P bcbfef3232264a4dbfa51874f4ac555e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:32:14.145561 12453 tablet_bootstrap.cc:492] T ee7b48a88ff84bf2beeb421986bb558b P 60525532c20a4c358974dc8e75b0596d: No bootstrap required, opened a new log
I20260619 20:32:14.145618 12453 ts_tablet_manager.cc:1397] T ee7b48a88ff84bf2beeb421986bb558b P 60525532c20a4c358974dc8e75b0596d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 20:32:14.145809 12453 raft_consensus.cc:348] T ee7b48a88ff84bf2beeb421986bb558b P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } }
I20260619 20:32:14.145889 12453 raft_consensus.cc:374] T ee7b48a88ff84bf2beeb421986bb558b P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:14.145910 12453 raft_consensus.cc:729] T ee7b48a88ff84bf2beeb421986bb558b P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60525532c20a4c358974dc8e75b0596d, State: Initialized, Role: FOLLOWER
I20260619 20:32:14.146083 12453 consensus_queue.cc:260] T ee7b48a88ff84bf2beeb421986bb558b P 60525532c20a4c358974dc8e75b0596d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } }
I20260619 20:32:14.146330 12453 ts_tablet_manager.cc:1428] T ee7b48a88ff84bf2beeb421986bb558b P 60525532c20a4c358974dc8e75b0596d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 20:32:14.146337 12462 tablet_replica.cc:439] T ee7b48a88ff84bf2beeb421986bb558b P 60525532c20a4c358974dc8e75b0596d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } }
I20260619 20:32:14.146535 12462 tablet_replica.cc:442] T ee7b48a88ff84bf2beeb421986bb558b P 60525532c20a4c358974dc8e75b0596d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:32:14.165158 12460 raft_consensus.cc:482] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:32:14.165272 12460 raft_consensus.cc:504] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } }
I20260619 20:32:14.165436 12460 leader_election.cc:288] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcbfef3232264a4dbfa51874f4ac555e (127.4.83.67:36719), 60525532c20a4c358974dc8e75b0596d (127.4.83.66:40003)
I20260619 20:32:14.165781 12378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee7b48a88ff84bf2beeb421986bb558b" candidate_uuid: "9db5cca45bf244168b2aa2cfd5454764" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcbfef3232264a4dbfa51874f4ac555e" is_pre_election: true
I20260619 20:32:14.165807 12236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee7b48a88ff84bf2beeb421986bb558b" candidate_uuid: "9db5cca45bf244168b2aa2cfd5454764" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60525532c20a4c358974dc8e75b0596d" is_pre_election: true
I20260619 20:32:14.165964 12236 raft_consensus.cc:2454] T ee7b48a88ff84bf2beeb421986bb558b P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9db5cca45bf244168b2aa2cfd5454764 in term 0.
I20260619 20:32:14.165874 12378 raft_consensus.cc:2454] T ee7b48a88ff84bf2beeb421986bb558b P bcbfef3232264a4dbfa51874f4ac555e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9db5cca45bf244168b2aa2cfd5454764 in term 0.
I20260619 20:32:14.166174 12031 leader_election.cc:302] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60525532c20a4c358974dc8e75b0596d, 9db5cca45bf244168b2aa2cfd5454764; no voters:
I20260619 20:32:14.166352 12460 raft_consensus.cc:2781] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:32:14.166409 12460 raft_consensus.cc:482] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:32:14.166424 12460 raft_consensus.cc:3037] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:14.167286 12460 raft_consensus.cc:504] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } }
I20260619 20:32:14.167451 12460 leader_election.cc:288] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764 [CANDIDATE]: Term 1 election: Requested vote from peers bcbfef3232264a4dbfa51874f4ac555e (127.4.83.67:36719), 60525532c20a4c358974dc8e75b0596d (127.4.83.66:40003)
I20260619 20:32:14.167670 12378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee7b48a88ff84bf2beeb421986bb558b" candidate_uuid: "9db5cca45bf244168b2aa2cfd5454764" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcbfef3232264a4dbfa51874f4ac555e"
I20260619 20:32:14.167702 12236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee7b48a88ff84bf2beeb421986bb558b" candidate_uuid: "9db5cca45bf244168b2aa2cfd5454764" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60525532c20a4c358974dc8e75b0596d"
I20260619 20:32:14.167752 12378 raft_consensus.cc:3037] T ee7b48a88ff84bf2beeb421986bb558b P bcbfef3232264a4dbfa51874f4ac555e [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:14.167776 12236 raft_consensus.cc:3037] T ee7b48a88ff84bf2beeb421986bb558b P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:14.168658 12378 raft_consensus.cc:2454] T ee7b48a88ff84bf2beeb421986bb558b P bcbfef3232264a4dbfa51874f4ac555e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9db5cca45bf244168b2aa2cfd5454764 in term 1.
I20260619 20:32:14.168751 12236 raft_consensus.cc:2454] T ee7b48a88ff84bf2beeb421986bb558b P 60525532c20a4c358974dc8e75b0596d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9db5cca45bf244168b2aa2cfd5454764 in term 1.
I20260619 20:32:14.168885 12030 leader_election.cc:302] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9db5cca45bf244168b2aa2cfd5454764, bcbfef3232264a4dbfa51874f4ac555e; no voters:
I20260619 20:32:14.169015 12460 raft_consensus.cc:2781] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:32:14.169090 12460 raft_consensus.cc:686] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764 [term 1 LEADER]: Becoming Leader. State: Replica: 9db5cca45bf244168b2aa2cfd5454764, State: Running, Role: LEADER
I20260619 20:32:14.169150 12460 consensus_queue.cc:237] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } }
I20260619 20:32:14.169705 12482 tablet_replica.cc:439] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764: TxnStatusTablet state changed. Reason: New leader 9db5cca45bf244168b2aa2cfd5454764. Latest consensus state: current_term: 1 leader_uuid: "9db5cca45bf244168b2aa2cfd5454764" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } }
I20260619 20:32:14.169847 12482 tablet_replica.cc:442] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764: This TxnStatusTablet replica's current role is: LEADER
I20260619 20:32:14.170123 12494 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 20:32:14.170121 11939 catalog_manager.cc:5495] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9db5cca45bf244168b2aa2cfd5454764 (127.4.83.65). New cstate: current_term: 1 leader_uuid: "9db5cca45bf244168b2aa2cfd5454764" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } health_report { overall_health: UNKNOWN } } }
I20260619 20:32:14.170281 12494 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 20:32:14.180678 12489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:14.177888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:39085 (local address 127.0.0.1:48782)
0619 20:32:14.177986 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:14.177997 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:14.178044 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:14.178240 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:14.178241 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:14.178243 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:32:14.178312 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:14.178314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.178998 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.178999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:14.179625 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:14.179627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.180066 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.180067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:14.180140 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:14.180165 (+ 25us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:32:14.180463 (+ 298us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:32:14.180474 (+ 11us) client_negotiation.cc:769] Sending connection context
0619 20:32:14.180527 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 20:32:14.180597 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260619 20:32:14.180960 12451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:14.177996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48782 (local address 127.4.83.65:39085)
0619 20:32:14.178076 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:14.178079 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:14.178102 (+ 23us) server_negotiation.cc:407] Connection header received
0619 20:32:14.178145 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:14.178147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:14.178170 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:14.178226 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:32:14.178368 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.178910 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.179715 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.179995 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.180062 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:14.180259 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:32:14.180403 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:32:14.180474 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:14.180536 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 20:32:14.180565 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 20:32:14.182478 12378 raft_consensus.cc:1261] T ee7b48a88ff84bf2beeb421986bb558b P bcbfef3232264a4dbfa51874f4ac555e [term 1 FOLLOWER]: Refusing update from remote peer 9db5cca45bf244168b2aa2cfd5454764: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:32:14.182577 12236 raft_consensus.cc:1261] T ee7b48a88ff84bf2beeb421986bb558b P 60525532c20a4c358974dc8e75b0596d [term 1 FOLLOWER]: Refusing update from remote peer 9db5cca45bf244168b2aa2cfd5454764: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:32:14.182964 12482 consensus_queue.cc:1035] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:32:14.183159 12460 consensus_queue.cc:1035] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:32:14.184413 12462 tablet_replica.cc:439] T ee7b48a88ff84bf2beeb421986bb558b P 60525532c20a4c358974dc8e75b0596d: TxnStatusTablet state changed. Reason: New leader 9db5cca45bf244168b2aa2cfd5454764. Latest consensus state: current_term: 1 leader_uuid: "9db5cca45bf244168b2aa2cfd5454764" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } }
I20260619 20:32:14.184523 12462 tablet_replica.cc:442] T ee7b48a88ff84bf2beeb421986bb558b P 60525532c20a4c358974dc8e75b0596d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:32:14.184924 12459 tablet_replica.cc:439] T ee7b48a88ff84bf2beeb421986bb558b P bcbfef3232264a4dbfa51874f4ac555e: TxnStatusTablet state changed. Reason: New leader 9db5cca45bf244168b2aa2cfd5454764. Latest consensus state: current_term: 1 leader_uuid: "9db5cca45bf244168b2aa2cfd5454764" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } }
I20260619 20:32:14.184976 12482 tablet_replica.cc:439] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9db5cca45bf244168b2aa2cfd5454764" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } }
I20260619 20:32:14.185050 12459 tablet_replica.cc:442] T ee7b48a88ff84bf2beeb421986bb558b P bcbfef3232264a4dbfa51874f4ac555e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:32:14.185065 12482 tablet_replica.cc:442] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764: This TxnStatusTablet replica's current role is: LEADER
I20260619 20:32:14.186175 12460 tablet_replica.cc:439] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9db5cca45bf244168b2aa2cfd5454764" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } }
I20260619 20:32:14.186261 12460 tablet_replica.cc:442] T ee7b48a88ff84bf2beeb421986bb558b P 9db5cca45bf244168b2aa2cfd5454764: This TxnStatusTablet replica's current role is: LEADER
I20260619 20:32:14.186434 12459 tablet_replica.cc:439] T ee7b48a88ff84bf2beeb421986bb558b P bcbfef3232264a4dbfa51874f4ac555e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9db5cca45bf244168b2aa2cfd5454764" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } }
I20260619 20:32:14.186556 12459 tablet_replica.cc:442] T ee7b48a88ff84bf2beeb421986bb558b P bcbfef3232264a4dbfa51874f4ac555e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:32:14.186651 12465 mvcc.cc:205] Tried to move back new op lower bound from 7298667045608402944 to 7298667045557940224. Current Snapshot: MvccSnapshot[applied={T|T < 7298667045608402944 or (T in {7298667045608402944})}]
I20260619 20:32:14.186964 12462 tablet_replica.cc:439] T ee7b48a88ff84bf2beeb421986bb558b P 60525532c20a4c358974dc8e75b0596d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9db5cca45bf244168b2aa2cfd5454764" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } }
I20260619 20:32:14.187053 12462 tablet_replica.cc:442] T ee7b48a88ff84bf2beeb421986bb558b P 60525532c20a4c358974dc8e75b0596d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:32:14.190984 12451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:14.187917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48784 (local address 127.4.83.65:39085)
0619 20:32:14.188004 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:14.188007 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:14.188050 (+ 43us) server_negotiation.cc:407] Connection header received
0619 20:32:14.188127 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:14.188128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:14.188139 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:14.188188 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:32:14.188362 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.188881 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.190045 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.190261 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.190337 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:14.190580 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:32:14.190633 (+ 53us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:32:14.190719 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:14.190805 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 20:32:14.190870 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 20:32:14.198526 12451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:14.194947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:38481 (local address 127.4.83.65:39085)
0619 20:32:14.195012 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:14.195015 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:14.195248 (+ 233us) server_negotiation.cc:407] Connection header received
0619 20:32:14.195278 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:14.195279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:14.195291 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:14.195335 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:32:14.195493 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.196219 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.197606 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.197846 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.197968 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:14.198212 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:32:14.198249 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:32:14.198286 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:14.198380 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 20:32:14.198406 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 20:32:14.198674 12501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:14.194842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:39085 (local address 127.4.83.65:38481)
0619 20:32:14.195095 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:14.195111 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:14.195197 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:14.195353 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:14.195354 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:14.195355 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:32:14.195431 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:14.195433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.196357 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.196358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:14.197479 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:14.197482 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.197995 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.197996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:14.198082 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:14.198113 (+ 31us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:32:14.198309 (+ 196us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:32:14.198311 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:32:14.198374 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 20:32:14.198427 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":100,"threads_started":1}
I20260619 20:32:14.202527 12236 raft_consensus.cc:1261] T 676c95bdc427474dadbb7ed9fabc13a9 P 60525532c20a4c358974dc8e75b0596d [term 1 FOLLOWER]: Refusing update from remote peer 9db5cca45bf244168b2aa2cfd5454764: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:32:14.202747 12378 raft_consensus.cc:1261] T 676c95bdc427474dadbb7ed9fabc13a9 P bcbfef3232264a4dbfa51874f4ac555e [term 1 FOLLOWER]: Refusing update from remote peer 9db5cca45bf244168b2aa2cfd5454764: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:32:14.202862 12460 consensus_queue.cc:1035] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:32:14.203217 12460 consensus_queue.cc:1035] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:32:14.221624 11940 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:32:14.221777 11940 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:32:14.222604 11940 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 74439e8536d84e1884988644eba75b84: Sending DeleteTablet for 3 replicas of tablet 676c95bdc427474dadbb7ed9fabc13a9
I20260619 20:32:14.222961 12078 tablet_service.cc:1514] Processing DeleteTablet for tablet 676c95bdc427474dadbb7ed9fabc13a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:32:14 UTC) from {username='oryx', principal='oryx/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:48768
I20260619 20:32:14.222999 12218 tablet_service.cc:1514] Processing DeleteTablet for tablet 676c95bdc427474dadbb7ed9fabc13a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:32:14 UTC) from {username='oryx', principal='oryx/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:38276
I20260619 20:32:14.223021 12358 tablet_service.cc:1514] Processing DeleteTablet for tablet 676c95bdc427474dadbb7ed9fabc13a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:32:14 UTC) from {username='oryx', principal='oryx/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:53124
I20260619 20:32:14.223110 12474 tablet_replica.cc:330] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764: stopping tablet replica
I20260619 20:32:14.223165 12474 raft_consensus.cc:2229] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:32:14.223155 12476 tablet_replica.cc:330] T 676c95bdc427474dadbb7ed9fabc13a9 P 60525532c20a4c358974dc8e75b0596d: stopping tablet replica
I20260619 20:32:14.223235 12476 raft_consensus.cc:2229] T 676c95bdc427474dadbb7ed9fabc13a9 P 60525532c20a4c358974dc8e75b0596d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:32:14.223282 12476 raft_consensus.cc:2258] T 676c95bdc427474dadbb7ed9fabc13a9 P 60525532c20a4c358974dc8e75b0596d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:32:14.223310 12474 raft_consensus.cc:2258] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:32:14.223361 12475 tablet_replica.cc:330] T 676c95bdc427474dadbb7ed9fabc13a9 P bcbfef3232264a4dbfa51874f4ac555e: stopping tablet replica
I20260619 20:32:14.223428 12475 raft_consensus.cc:2229] T 676c95bdc427474dadbb7ed9fabc13a9 P bcbfef3232264a4dbfa51874f4ac555e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:32:14.223490 12475 raft_consensus.cc:2258] T 676c95bdc427474dadbb7ed9fabc13a9 P bcbfef3232264a4dbfa51874f4ac555e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:32:14.223917 12476 ts_tablet_manager.cc:1905] T 676c95bdc427474dadbb7ed9fabc13a9 P 60525532c20a4c358974dc8e75b0596d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:32:14.223997 12475 ts_tablet_manager.cc:1905] T 676c95bdc427474dadbb7ed9fabc13a9 P bcbfef3232264a4dbfa51874f4ac555e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:32:14.224035 12474 ts_tablet_manager.cc:1905] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:32:14.226048 12476 ts_tablet_manager.cc:1918] T 676c95bdc427474dadbb7ed9fabc13a9 P 60525532c20a4c358974dc8e75b0596d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:32:14.226080 12475 ts_tablet_manager.cc:1918] T 676c95bdc427474dadbb7ed9fabc13a9 P bcbfef3232264a4dbfa51874f4ac555e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:32:14.226121 12476 log.cc:1196] T 676c95bdc427474dadbb7ed9fabc13a9 P 60525532c20a4c358974dc8e75b0596d: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-1/wal/wals/676c95bdc427474dadbb7ed9fabc13a9
I20260619 20:32:14.226161 12475 log.cc:1196] T 676c95bdc427474dadbb7ed9fabc13a9 P bcbfef3232264a4dbfa51874f4ac555e: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-2/wal/wals/676c95bdc427474dadbb7ed9fabc13a9
I20260619 20:32:14.226370 12476 ts_tablet_manager.cc:1939] T 676c95bdc427474dadbb7ed9fabc13a9 P 60525532c20a4c358974dc8e75b0596d: Deleting consensus metadata
I20260619 20:32:14.226447 12475 ts_tablet_manager.cc:1939] T 676c95bdc427474dadbb7ed9fabc13a9 P bcbfef3232264a4dbfa51874f4ac555e: Deleting consensus metadata
I20260619 20:32:14.226821 12474 ts_tablet_manager.cc:1918] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:32:14.226886 12474 log.cc:1196] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-0/wal/wals/676c95bdc427474dadbb7ed9fabc13a9
I20260619 20:32:14.226928 11926 catalog_manager.cc:4841] TS bcbfef3232264a4dbfa51874f4ac555e (127.4.83.67:36719): tablet 676c95bdc427474dadbb7ed9fabc13a9 (table test-table [id=4b84a09d99c74dd4812e2ef610a5412a]) successfully deleted
I20260619 20:32:14.227084 12474 ts_tablet_manager.cc:1939] T 676c95bdc427474dadbb7ed9fabc13a9 P 9db5cca45bf244168b2aa2cfd5454764: Deleting consensus metadata
I20260619 20:32:14.227705 11927 catalog_manager.cc:4841] TS 9db5cca45bf244168b2aa2cfd5454764 (127.4.83.65:39085): tablet 676c95bdc427474dadbb7ed9fabc13a9 (table test-table [id=4b84a09d99c74dd4812e2ef610a5412a]) successfully deleted
I20260619 20:32:14.228813 11927 catalog_manager.cc:4841] TS 60525532c20a4c358974dc8e75b0596d (127.4.83.66:40003): tablet 676c95bdc427474dadbb7ed9fabc13a9 (table test-table [id=4b84a09d99c74dd4812e2ef610a5412a]) successfully deleted
Jun 19 20:32:14 dist-test-slave-4m58 krb5kdc[11895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 20:32:14 dist-test-slave-4m58 krb5kdc[11895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:14 dist-test-slave-4m58 krb5kdc[11895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
W20260619 20:32:14.254917 11990 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:14.247558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59928 (local address 127.4.83.126:43281)
0619 20:32:14.247630 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:14.247632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:14.247722 (+ 90us) server_negotiation.cc:407] Connection header received
0619 20:32:14.247793 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:14.247794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:14.247806 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:14.247921 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:14.248988 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.249722 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.250766 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.251061 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.251144 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:14.254840 (+ 3696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59928: BlockingRecv error: recv got EOF from 127.0.0.1:59928 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Jun 19 20:32:14 dist-test-slave-4m58 krb5kdc[11895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM
I20260619 20:32:14.266302 11990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:14.256277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59930 (local address 127.4.83.126:43281)
0619 20:32:14.256381 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:14.256384 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:14.256431 (+ 47us) server_negotiation.cc:407] Connection header received
0619 20:32:14.256523 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:14.256525 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:14.256537 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:14.256588 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:14.257143 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.257793 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.258703 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.259065 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.259149 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:14.262393 (+ 3244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:14.262395 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:14.262396 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:14.262409 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:14.264399 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:14.265045 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:14.265046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:14.265047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:14.265132 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:14.265479 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:14.265480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:14.265481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:14.265651 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:14.265776 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:14.266094 (+ 318us) server_negotiation.cc:299] Negotiation successful
0619 20:32:14.266202 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 20:32:14.268810 11940 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59930:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:32:14.269032 11940 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:32:14.271061 12358 tablet_service.cc:1467] Processing CreateTablet for tablet db9cadf18f19415f9fd620f19dade0d1 (DEFAULT_TABLE table=test-table [id=5f3ae87b7bd1408daf86f8acb55eaedd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:32:14.271147 12078 tablet_service.cc:1467] Processing CreateTablet for tablet db9cadf18f19415f9fd620f19dade0d1 (DEFAULT_TABLE table=test-table [id=5f3ae87b7bd1408daf86f8acb55eaedd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:32:14.271293 12358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db9cadf18f19415f9fd620f19dade0d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:14.271320 12078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db9cadf18f19415f9fd620f19dade0d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:14.271286 12218 tablet_service.cc:1467] Processing CreateTablet for tablet db9cadf18f19415f9fd620f19dade0d1 (DEFAULT_TABLE table=test-table [id=5f3ae87b7bd1408daf86f8acb55eaedd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:32:14.271461 12218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db9cadf18f19415f9fd620f19dade0d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:14.273196 12453 tablet_bootstrap.cc:492] T db9cadf18f19415f9fd620f19dade0d1 P 60525532c20a4c358974dc8e75b0596d: Bootstrap starting.
I20260619 20:32:14.273279 12454 tablet_bootstrap.cc:492] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764: Bootstrap starting.
I20260619 20:32:14.273263 12452 tablet_bootstrap.cc:492] T db9cadf18f19415f9fd620f19dade0d1 P bcbfef3232264a4dbfa51874f4ac555e: Bootstrap starting.
I20260619 20:32:14.273610 12453 tablet_bootstrap.cc:654] T db9cadf18f19415f9fd620f19dade0d1 P 60525532c20a4c358974dc8e75b0596d: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:14.273811 12454 tablet_bootstrap.cc:654] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:14.273835 12452 tablet_bootstrap.cc:654] T db9cadf18f19415f9fd620f19dade0d1 P bcbfef3232264a4dbfa51874f4ac555e: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:14.274669 12454 tablet_bootstrap.cc:492] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764: No bootstrap required, opened a new log
I20260619 20:32:14.274747 12454 ts_tablet_manager.cc:1397] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 20:32:14.274801 12452 tablet_bootstrap.cc:492] T db9cadf18f19415f9fd620f19dade0d1 P bcbfef3232264a4dbfa51874f4ac555e: No bootstrap required, opened a new log
I20260619 20:32:14.274850 12452 ts_tablet_manager.cc:1397] T db9cadf18f19415f9fd620f19dade0d1 P bcbfef3232264a4dbfa51874f4ac555e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260619 20:32:14.274930 12454 raft_consensus.cc:348] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } }
I20260619 20:32:14.275063 12454 raft_consensus.cc:374] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:14.275095 12454 raft_consensus.cc:729] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9db5cca45bf244168b2aa2cfd5454764, State: Initialized, Role: FOLLOWER
I20260619 20:32:14.275064 12452 raft_consensus.cc:348] T db9cadf18f19415f9fd620f19dade0d1 P bcbfef3232264a4dbfa51874f4ac555e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } }
I20260619 20:32:14.275158 12452 raft_consensus.cc:374] T db9cadf18f19415f9fd620f19dade0d1 P bcbfef3232264a4dbfa51874f4ac555e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:14.275183 12452 raft_consensus.cc:729] T db9cadf18f19415f9fd620f19dade0d1 P bcbfef3232264a4dbfa51874f4ac555e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcbfef3232264a4dbfa51874f4ac555e, State: Initialized, Role: FOLLOWER
I20260619 20:32:14.275175 12454 consensus_queue.cc:260] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } }
I20260619 20:32:14.275303 12454 ts_tablet_manager.cc:1428] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260619 20:32:14.275337 12452 consensus_queue.cc:260] T db9cadf18f19415f9fd620f19dade0d1 P bcbfef3232264a4dbfa51874f4ac555e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } }
I20260619 20:32:14.275498 12452 ts_tablet_manager.cc:1428] T db9cadf18f19415f9fd620f19dade0d1 P bcbfef3232264a4dbfa51874f4ac555e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 20:32:14.276175 12453 tablet_bootstrap.cc:492] T db9cadf18f19415f9fd620f19dade0d1 P 60525532c20a4c358974dc8e75b0596d: No bootstrap required, opened a new log
I20260619 20:32:14.276286 12453 ts_tablet_manager.cc:1397] T db9cadf18f19415f9fd620f19dade0d1 P 60525532c20a4c358974dc8e75b0596d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260619 20:32:14.276451 12453 raft_consensus.cc:348] T db9cadf18f19415f9fd620f19dade0d1 P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } }
I20260619 20:32:14.276523 12453 raft_consensus.cc:374] T db9cadf18f19415f9fd620f19dade0d1 P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:14.276536 12453 raft_consensus.cc:729] T db9cadf18f19415f9fd620f19dade0d1 P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60525532c20a4c358974dc8e75b0596d, State: Initialized, Role: FOLLOWER
I20260619 20:32:14.276582 12453 consensus_queue.cc:260] T db9cadf18f19415f9fd620f19dade0d1 P 60525532c20a4c358974dc8e75b0596d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } }
I20260619 20:32:14.276682 12453 ts_tablet_manager.cc:1428] T db9cadf18f19415f9fd620f19dade0d1 P 60525532c20a4c358974dc8e75b0596d: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 20:32:14.277508 12504 raft_consensus.cc:482] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:32:14.277587 12504 raft_consensus.cc:504] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } }
I20260619 20:32:14.277760 12504 leader_election.cc:288] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60525532c20a4c358974dc8e75b0596d (127.4.83.66:40003), bcbfef3232264a4dbfa51874f4ac555e (127.4.83.67:36719)
I20260619 20:32:14.277942 12236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db9cadf18f19415f9fd620f19dade0d1" candidate_uuid: "9db5cca45bf244168b2aa2cfd5454764" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60525532c20a4c358974dc8e75b0596d" is_pre_election: true
I20260619 20:32:14.278031 12378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db9cadf18f19415f9fd620f19dade0d1" candidate_uuid: "9db5cca45bf244168b2aa2cfd5454764" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcbfef3232264a4dbfa51874f4ac555e" is_pre_election: true
I20260619 20:32:14.278088 12236 raft_consensus.cc:2454] T db9cadf18f19415f9fd620f19dade0d1 P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9db5cca45bf244168b2aa2cfd5454764 in term 0.
I20260619 20:32:14.278126 12378 raft_consensus.cc:2454] T db9cadf18f19415f9fd620f19dade0d1 P bcbfef3232264a4dbfa51874f4ac555e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9db5cca45bf244168b2aa2cfd5454764 in term 0.
I20260619 20:32:14.278270 12031 leader_election.cc:302] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60525532c20a4c358974dc8e75b0596d, 9db5cca45bf244168b2aa2cfd5454764; no voters:
I20260619 20:32:14.278384 12504 raft_consensus.cc:2781] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:32:14.278419 12504 raft_consensus.cc:482] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:32:14.278431 12504 raft_consensus.cc:3037] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:14.279337 12504 raft_consensus.cc:504] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } }
I20260619 20:32:14.279506 12504 leader_election.cc:288] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764 [CANDIDATE]: Term 1 election: Requested vote from peers 60525532c20a4c358974dc8e75b0596d (127.4.83.66:40003), bcbfef3232264a4dbfa51874f4ac555e (127.4.83.67:36719)
I20260619 20:32:14.279721 12236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db9cadf18f19415f9fd620f19dade0d1" candidate_uuid: "9db5cca45bf244168b2aa2cfd5454764" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60525532c20a4c358974dc8e75b0596d"
I20260619 20:32:14.279722 12378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db9cadf18f19415f9fd620f19dade0d1" candidate_uuid: "9db5cca45bf244168b2aa2cfd5454764" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcbfef3232264a4dbfa51874f4ac555e"
I20260619 20:32:14.279822 12236 raft_consensus.cc:3037] T db9cadf18f19415f9fd620f19dade0d1 P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:14.279824 12378 raft_consensus.cc:3037] T db9cadf18f19415f9fd620f19dade0d1 P bcbfef3232264a4dbfa51874f4ac555e [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:14.280835 12378 raft_consensus.cc:2454] T db9cadf18f19415f9fd620f19dade0d1 P bcbfef3232264a4dbfa51874f4ac555e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9db5cca45bf244168b2aa2cfd5454764 in term 1.
I20260619 20:32:14.280835 12236 raft_consensus.cc:2454] T db9cadf18f19415f9fd620f19dade0d1 P 60525532c20a4c358974dc8e75b0596d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9db5cca45bf244168b2aa2cfd5454764 in term 1.
I20260619 20:32:14.281097 12031 leader_election.cc:302] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 60525532c20a4c358974dc8e75b0596d, 9db5cca45bf244168b2aa2cfd5454764, bcbfef3232264a4dbfa51874f4ac555e; no voters:
I20260619 20:32:14.281240 12504 raft_consensus.cc:2781] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:32:14.281289 12504 raft_consensus.cc:686] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764 [term 1 LEADER]: Becoming Leader. State: Replica: 9db5cca45bf244168b2aa2cfd5454764, State: Running, Role: LEADER
I20260619 20:32:14.281345 12504 consensus_queue.cc:237] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } }
I20260619 20:32:14.282151 11940 catalog_manager.cc:5495] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9db5cca45bf244168b2aa2cfd5454764 (127.4.83.65). New cstate: current_term: 1 leader_uuid: "9db5cca45bf244168b2aa2cfd5454764" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } health_report { overall_health: UNKNOWN } } }
I20260619 20:32:14.293342 12451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:14.290079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48796 (local address 127.4.83.65:39085)
0619 20:32:14.290156 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:14.290159 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:14.290171 (+ 12us) server_negotiation.cc:407] Connection header received
0619 20:32:14.290201 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:14.290202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:14.290216 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:14.290310 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:32:14.290528 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.291322 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.292347 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.292661 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.292756 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:14.292963 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:32:14.293005 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:32:14.293077 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:14.293215 (+ 138us) server_negotiation.cc:299] Negotiation successful
0619 20:32:14.293237 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 20:32:14.294373 12236 raft_consensus.cc:1261] T db9cadf18f19415f9fd620f19dade0d1 P 60525532c20a4c358974dc8e75b0596d [term 1 FOLLOWER]: Refusing update from remote peer 9db5cca45bf244168b2aa2cfd5454764: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:32:14.294399 12378 raft_consensus.cc:1261] T db9cadf18f19415f9fd620f19dade0d1 P bcbfef3232264a4dbfa51874f4ac555e [term 1 FOLLOWER]: Refusing update from remote peer 9db5cca45bf244168b2aa2cfd5454764: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:32:14.294692 12504 consensus_queue.cc:1035] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:32:14.294831 12482 consensus_queue.cc:1035] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:32:14.298484 11939 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:32:14.298700 11939 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:32:14.299624 11939 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 74439e8536d84e1884988644eba75b84: Sending DeleteTablet for 3 replicas of tablet db9cadf18f19415f9fd620f19dade0d1
I20260619 20:32:14.300020 12078 tablet_service.cc:1514] Processing DeleteTablet for tablet db9cadf18f19415f9fd620f19dade0d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:32:14 UTC) from {username='oryx', principal='oryx/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:48768
I20260619 20:32:14.300100 12218 tablet_service.cc:1514] Processing DeleteTablet for tablet db9cadf18f19415f9fd620f19dade0d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:32:14 UTC) from {username='oryx', principal='oryx/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:38276
I20260619 20:32:14.300171 12474 tablet_replica.cc:330] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764: stopping tablet replica
I20260619 20:32:14.300203 12476 tablet_replica.cc:330] T db9cadf18f19415f9fd620f19dade0d1 P 60525532c20a4c358974dc8e75b0596d: stopping tablet replica
I20260619 20:32:14.300230 12474 raft_consensus.cc:2229] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:32:14.300257 12476 raft_consensus.cc:2229] T db9cadf18f19415f9fd620f19dade0d1 P 60525532c20a4c358974dc8e75b0596d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:32:14.300308 12476 raft_consensus.cc:2258] T db9cadf18f19415f9fd620f19dade0d1 P 60525532c20a4c358974dc8e75b0596d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:32:14.300308 12474 raft_consensus.cc:2258] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:32:14.300252 12358 tablet_service.cc:1514] Processing DeleteTablet for tablet db9cadf18f19415f9fd620f19dade0d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:32:14 UTC) from {username='oryx', principal='oryx/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:53124
I20260619 20:32:14.300397 12475 tablet_replica.cc:330] T db9cadf18f19415f9fd620f19dade0d1 P bcbfef3232264a4dbfa51874f4ac555e: stopping tablet replica
I20260619 20:32:14.300498 12475 raft_consensus.cc:2229] T db9cadf18f19415f9fd620f19dade0d1 P bcbfef3232264a4dbfa51874f4ac555e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:32:14.300559 12475 raft_consensus.cc:2258] T db9cadf18f19415f9fd620f19dade0d1 P bcbfef3232264a4dbfa51874f4ac555e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:32:14.300794 12474 ts_tablet_manager.cc:1905] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:32:14.300938 11939 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59930:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:32:14.301218 11939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:32:14.302806 12078 tablet_service.cc:1467] Processing CreateTablet for tablet 120e6e4fab59474cb2a8bb365a26be49 (DEFAULT_TABLE table=test-table [id=c53ab449bda440149614ba5346ce46ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:32:14.302794 12358 tablet_service.cc:1467] Processing CreateTablet for tablet 120e6e4fab59474cb2a8bb365a26be49 (DEFAULT_TABLE table=test-table [id=c53ab449bda440149614ba5346ce46ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:32:14.302940 12474 ts_tablet_manager.cc:1918] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:32:14.302992 12474 log.cc:1196] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-0/wal/wals/db9cadf18f19415f9fd620f19dade0d1
I20260619 20:32:14.303081 12358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 120e6e4fab59474cb2a8bb365a26be49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:14.303158 12218 tablet_service.cc:1467] Processing CreateTablet for tablet 120e6e4fab59474cb2a8bb365a26be49 (DEFAULT_TABLE table=test-table [id=c53ab449bda440149614ba5346ce46ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:32:14.303220 12474 ts_tablet_manager.cc:1939] T db9cadf18f19415f9fd620f19dade0d1 P 9db5cca45bf244168b2aa2cfd5454764: Deleting consensus metadata
I20260619 20:32:14.303287 12218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 120e6e4fab59474cb2a8bb365a26be49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:14.302943 12078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 120e6e4fab59474cb2a8bb365a26be49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:14.304930 12453 tablet_bootstrap.cc:492] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d: Bootstrap starting.
I20260619 20:32:14.305126 12454 tablet_bootstrap.cc:492] T 120e6e4fab59474cb2a8bb365a26be49 P 9db5cca45bf244168b2aa2cfd5454764: Bootstrap starting.
I20260619 20:32:14.305330 12453 tablet_bootstrap.cc:654] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:14.305655 12476 ts_tablet_manager.cc:1905] T db9cadf18f19415f9fd620f19dade0d1 P 60525532c20a4c358974dc8e75b0596d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:32:14.306003 12454 tablet_bootstrap.cc:654] T 120e6e4fab59474cb2a8bb365a26be49 P 9db5cca45bf244168b2aa2cfd5454764: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:14.306109 11927 catalog_manager.cc:4841] TS 9db5cca45bf244168b2aa2cfd5454764 (127.4.83.65:39085): tablet db9cadf18f19415f9fd620f19dade0d1 (table test-table [id=5f3ae87b7bd1408daf86f8acb55eaedd]) successfully deleted
I20260619 20:32:14.306442 12452 tablet_bootstrap.cc:492] T 120e6e4fab59474cb2a8bb365a26be49 P bcbfef3232264a4dbfa51874f4ac555e: Bootstrap starting.
I20260619 20:32:14.306924 12453 tablet_bootstrap.cc:492] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d: No bootstrap required, opened a new log
I20260619 20:32:14.306975 12453 ts_tablet_manager.cc:1397] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 20:32:14.307118 12453 raft_consensus.cc:348] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } }
I20260619 20:32:14.307202 12453 raft_consensus.cc:374] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:14.307224 12453 raft_consensus.cc:729] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60525532c20a4c358974dc8e75b0596d, State: Initialized, Role: FOLLOWER
I20260619 20:32:14.307278 12453 consensus_queue.cc:260] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } }
I20260619 20:32:14.307314 12452 tablet_bootstrap.cc:654] T 120e6e4fab59474cb2a8bb365a26be49 P bcbfef3232264a4dbfa51874f4ac555e: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:14.307394 12476 ts_tablet_manager.cc:1918] T db9cadf18f19415f9fd620f19dade0d1 P 60525532c20a4c358974dc8e75b0596d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:32:14.307453 12476 log.cc:1196] T db9cadf18f19415f9fd620f19dade0d1 P 60525532c20a4c358974dc8e75b0596d: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-1/wal/wals/db9cadf18f19415f9fd620f19dade0d1
I20260619 20:32:14.307665 12476 ts_tablet_manager.cc:1939] T db9cadf18f19415f9fd620f19dade0d1 P 60525532c20a4c358974dc8e75b0596d: Deleting consensus metadata
I20260619 20:32:14.307391 12453 ts_tablet_manager.cc:1428] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:14.308210 11927 catalog_manager.cc:4841] TS 60525532c20a4c358974dc8e75b0596d (127.4.83.66:40003): tablet db9cadf18f19415f9fd620f19dade0d1 (table test-table [id=5f3ae87b7bd1408daf86f8acb55eaedd]) successfully deleted
I20260619 20:32:14.309058 12454 tablet_bootstrap.cc:492] T 120e6e4fab59474cb2a8bb365a26be49 P 9db5cca45bf244168b2aa2cfd5454764: No bootstrap required, opened a new log
I20260619 20:32:14.309091 12475 ts_tablet_manager.cc:1905] T db9cadf18f19415f9fd620f19dade0d1 P bcbfef3232264a4dbfa51874f4ac555e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:32:14.309135 12454 ts_tablet_manager.cc:1397] T 120e6e4fab59474cb2a8bb365a26be49 P 9db5cca45bf244168b2aa2cfd5454764: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 20:32:14.309206 12452 tablet_bootstrap.cc:492] T 120e6e4fab59474cb2a8bb365a26be49 P bcbfef3232264a4dbfa51874f4ac555e: No bootstrap required, opened a new log
I20260619 20:32:14.309264 12452 ts_tablet_manager.cc:1397] T 120e6e4fab59474cb2a8bb365a26be49 P bcbfef3232264a4dbfa51874f4ac555e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260619 20:32:14.309324 12454 raft_consensus.cc:348] T 120e6e4fab59474cb2a8bb365a26be49 P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } }
I20260619 20:32:14.309408 12454 raft_consensus.cc:374] T 120e6e4fab59474cb2a8bb365a26be49 P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:14.309423 12454 raft_consensus.cc:729] T 120e6e4fab59474cb2a8bb365a26be49 P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9db5cca45bf244168b2aa2cfd5454764, State: Initialized, Role: FOLLOWER
I20260619 20:32:14.309473 12454 consensus_queue.cc:260] T 120e6e4fab59474cb2a8bb365a26be49 P 9db5cca45bf244168b2aa2cfd5454764 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } }
I20260619 20:32:14.309566 12454 ts_tablet_manager.cc:1428] T 120e6e4fab59474cb2a8bb365a26be49 P 9db5cca45bf244168b2aa2cfd5454764: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:14.309506 12452 raft_consensus.cc:348] T 120e6e4fab59474cb2a8bb365a26be49 P bcbfef3232264a4dbfa51874f4ac555e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } }
I20260619 20:32:14.309610 12452 raft_consensus.cc:374] T 120e6e4fab59474cb2a8bb365a26be49 P bcbfef3232264a4dbfa51874f4ac555e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:14.309633 12452 raft_consensus.cc:729] T 120e6e4fab59474cb2a8bb365a26be49 P bcbfef3232264a4dbfa51874f4ac555e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcbfef3232264a4dbfa51874f4ac555e, State: Initialized, Role: FOLLOWER
I20260619 20:32:14.309689 12452 consensus_queue.cc:260] T 120e6e4fab59474cb2a8bb365a26be49 P bcbfef3232264a4dbfa51874f4ac555e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } }
I20260619 20:32:14.309782 12452 ts_tablet_manager.cc:1428] T 120e6e4fab59474cb2a8bb365a26be49 P bcbfef3232264a4dbfa51874f4ac555e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:14.311043 12475 ts_tablet_manager.cc:1918] T db9cadf18f19415f9fd620f19dade0d1 P bcbfef3232264a4dbfa51874f4ac555e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:32:14.311120 12475 log.cc:1196] T db9cadf18f19415f9fd620f19dade0d1 P bcbfef3232264a4dbfa51874f4ac555e: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-2/wal/wals/db9cadf18f19415f9fd620f19dade0d1
I20260619 20:32:14.311298 12475 ts_tablet_manager.cc:1939] T db9cadf18f19415f9fd620f19dade0d1 P bcbfef3232264a4dbfa51874f4ac555e: Deleting consensus metadata
I20260619 20:32:14.312041 11926 catalog_manager.cc:4841] TS bcbfef3232264a4dbfa51874f4ac555e (127.4.83.67:36719): tablet db9cadf18f19415f9fd620f19dade0d1 (table test-table [id=5f3ae87b7bd1408daf86f8acb55eaedd]) successfully deleted
I20260619 20:32:14.593369 12462 raft_consensus.cc:482] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:32:14.593523 12462 raft_consensus.cc:504] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } }
I20260619 20:32:14.593750 12462 leader_election.cc:288] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9db5cca45bf244168b2aa2cfd5454764 (127.4.83.65:39085), bcbfef3232264a4dbfa51874f4ac555e (127.4.83.67:36719)
I20260619 20:32:14.594069 12378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "120e6e4fab59474cb2a8bb365a26be49" candidate_uuid: "60525532c20a4c358974dc8e75b0596d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcbfef3232264a4dbfa51874f4ac555e" is_pre_election: true
I20260619 20:32:14.594090 12098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "120e6e4fab59474cb2a8bb365a26be49" candidate_uuid: "60525532c20a4c358974dc8e75b0596d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9db5cca45bf244168b2aa2cfd5454764" is_pre_election: true
I20260619 20:32:14.594182 12378 raft_consensus.cc:2454] T 120e6e4fab59474cb2a8bb365a26be49 P bcbfef3232264a4dbfa51874f4ac555e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60525532c20a4c358974dc8e75b0596d in term 0.
I20260619 20:32:14.594209 12098 raft_consensus.cc:2454] T 120e6e4fab59474cb2a8bb365a26be49 P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60525532c20a4c358974dc8e75b0596d in term 0.
I20260619 20:32:14.594406 12172 leader_election.cc:302] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60525532c20a4c358974dc8e75b0596d, 9db5cca45bf244168b2aa2cfd5454764; no voters:
I20260619 20:32:14.594561 12462 raft_consensus.cc:2781] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:32:14.594620 12462 raft_consensus.cc:482] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:32:14.594635 12462 raft_consensus.cc:3037] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:14.595765 12462 raft_consensus.cc:504] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } }
I20260619 20:32:14.595980 12462 leader_election.cc:288] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d [CANDIDATE]: Term 1 election: Requested vote from peers 9db5cca45bf244168b2aa2cfd5454764 (127.4.83.65:39085), bcbfef3232264a4dbfa51874f4ac555e (127.4.83.67:36719)
I20260619 20:32:14.596187 12378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "120e6e4fab59474cb2a8bb365a26be49" candidate_uuid: "60525532c20a4c358974dc8e75b0596d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcbfef3232264a4dbfa51874f4ac555e"
I20260619 20:32:14.596201 12098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "120e6e4fab59474cb2a8bb365a26be49" candidate_uuid: "60525532c20a4c358974dc8e75b0596d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9db5cca45bf244168b2aa2cfd5454764"
I20260619 20:32:14.596278 12378 raft_consensus.cc:3037] T 120e6e4fab59474cb2a8bb365a26be49 P bcbfef3232264a4dbfa51874f4ac555e [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:14.596282 12098 raft_consensus.cc:3037] T 120e6e4fab59474cb2a8bb365a26be49 P 9db5cca45bf244168b2aa2cfd5454764 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:14.597034 12378 raft_consensus.cc:2454] T 120e6e4fab59474cb2a8bb365a26be49 P bcbfef3232264a4dbfa51874f4ac555e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60525532c20a4c358974dc8e75b0596d in term 1.
I20260619 20:32:14.597149 12098 raft_consensus.cc:2454] T 120e6e4fab59474cb2a8bb365a26be49 P 9db5cca45bf244168b2aa2cfd5454764 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60525532c20a4c358974dc8e75b0596d in term 1.
I20260619 20:32:14.597273 12171 leader_election.cc:302] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60525532c20a4c358974dc8e75b0596d, bcbfef3232264a4dbfa51874f4ac555e; no voters:
I20260619 20:32:14.597376 12462 raft_consensus.cc:2781] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:32:14.597498 12462 raft_consensus.cc:686] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d [term 1 LEADER]: Becoming Leader. State: Replica: 60525532c20a4c358974dc8e75b0596d, State: Running, Role: LEADER
I20260619 20:32:14.597556 12462 consensus_queue.cc:237] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } }
I20260619 20:32:14.598229 11940 catalog_manager.cc:5495] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d reported cstate change: term changed from 0 to 1, leader changed from <none> to 60525532c20a4c358974dc8e75b0596d (127.4.83.66). New cstate: current_term: 1 leader_uuid: "60525532c20a4c358974dc8e75b0596d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60525532c20a4c358974dc8e75b0596d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 } health_report { overall_health: UNKNOWN } } }
I20260619 20:32:14.673311 12530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:14.670778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38292 (local address 127.4.83.66:40003)
0619 20:32:14.670993 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:14.670995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:14.671005 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:32:14.671029 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:14.671030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:14.671042 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:14.671114 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:32:14.671262 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.671793 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.672611 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.672829 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.672899 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:14.673092 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:32:14.673124 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:32:14.673166 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:14.673220 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 20:32:14.673238 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":122,"threads_started":1}
I20260619 20:32:14.678694 12451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:14.676227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:58773 (local address 127.4.83.65:39085)
0619 20:32:14.676280 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:14.676283 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:14.676428 (+ 145us) server_negotiation.cc:407] Connection header received
0619 20:32:14.676489 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:14.676490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:14.676504 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:14.676579 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:32:14.676732 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.677216 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.678018 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.678216 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.678275 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:14.678469 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:32:14.678501 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:32:14.678558 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:14.678610 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 20:32:14.678629 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 20:32:14.678718 12533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:14.676124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:39085 (local address 127.4.83.66:58773)
0619 20:32:14.676322 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:14.676335 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:14.676402 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:14.676591 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:14.676591 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:14.676594 (+ 3us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:32:14.676661 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:14.676663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.677277 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.677278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:14.677933 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:14.677935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.678295 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.678296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:14.678340 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:14.678356 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:32:14.678566 (+ 210us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:32:14.678567 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 20:32:14.678596 (+ 29us) client_negotiation.cc:240] Negotiation successful
0619 20:32:14.678639 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":101,"threads_started":1}
I20260619 20:32:14.681214 12098 raft_consensus.cc:1261] T 120e6e4fab59474cb2a8bb365a26be49 P 9db5cca45bf244168b2aa2cfd5454764 [term 1 FOLLOWER]: Refusing update from remote peer 60525532c20a4c358974dc8e75b0596d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:32:14.681226 12378 raft_consensus.cc:1261] T 120e6e4fab59474cb2a8bb365a26be49 P bcbfef3232264a4dbfa51874f4ac555e [term 1 FOLLOWER]: Refusing update from remote peer 60525532c20a4c358974dc8e75b0596d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:32:14.681483 12462 consensus_queue.cc:1035] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9db5cca45bf244168b2aa2cfd5454764" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 39085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:32:14.681607 12528 consensus_queue.cc:1035] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcbfef3232264a4dbfa51874f4ac555e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 36719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:32:14.689278 12530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:14.686737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:45851 (local address 127.4.83.66:40003)
0619 20:32:14.686776 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:14.686778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:14.686788 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:32:14.686852 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:14.686853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:14.686866 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:14.686922 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:32:14.687059 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.687648 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.688509 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.688731 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.688807 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:14.688975 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:32:14.689013 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:32:14.689063 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:14.689171 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 20:32:14.689198 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 20:32:14.689318 12501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:14.686681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:40003 (local address 127.4.83.65:45851)
0619 20:32:14.686735 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:14.686748 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:14.686791 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:14.686936 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:14.686937 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:14.686939 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:32:14.687004 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:14.687006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.687731 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.687732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:14.688430 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:14.688433 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:14.688814 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:14.688815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:14.688873 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:14.688883 (+ 10us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:32:14.689112 (+ 229us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:32:14.689114 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:32:14.689151 (+ 37us) client_negotiation.cc:240] Negotiation successful
0619 20:32:14.689212 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260619 20:32:14.696386 11939 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:32:14.696517 11939 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:32:14.697320 11939 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 74439e8536d84e1884988644eba75b84: Sending DeleteTablet for 3 replicas of tablet 120e6e4fab59474cb2a8bb365a26be49
I20260619 20:32:14.697609 12218 tablet_service.cc:1514] Processing DeleteTablet for tablet 120e6e4fab59474cb2a8bb365a26be49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:32:14 UTC) from {username='oryx', principal='oryx/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:38276
I20260619 20:32:14.697626 12078 tablet_service.cc:1514] Processing DeleteTablet for tablet 120e6e4fab59474cb2a8bb365a26be49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:32:14 UTC) from {username='oryx', principal='oryx/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:48768
I20260619 20:32:14.697626 12358 tablet_service.cc:1514] Processing DeleteTablet for tablet 120e6e4fab59474cb2a8bb365a26be49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:32:14 UTC) from {username='oryx', principal='oryx/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:53124
I20260619 20:32:14.697723 12476 tablet_replica.cc:330] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d: stopping tablet replica
I20260619 20:32:14.697748 12475 tablet_replica.cc:330] T 120e6e4fab59474cb2a8bb365a26be49 P bcbfef3232264a4dbfa51874f4ac555e: stopping tablet replica
I20260619 20:32:14.697783 12476 raft_consensus.cc:2229] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:32:14.697798 12475 raft_consensus.cc:2229] T 120e6e4fab59474cb2a8bb365a26be49 P bcbfef3232264a4dbfa51874f4ac555e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:32:14.697836 12475 raft_consensus.cc:2258] T 120e6e4fab59474cb2a8bb365a26be49 P bcbfef3232264a4dbfa51874f4ac555e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:32:14.697868 12476 raft_consensus.cc:2258] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:32:14.697957 12474 tablet_replica.cc:330] T 120e6e4fab59474cb2a8bb365a26be49 P 9db5cca45bf244168b2aa2cfd5454764: stopping tablet replica
I20260619 20:32:14.698030 12474 raft_consensus.cc:2229] T 120e6e4fab59474cb2a8bb365a26be49 P 9db5cca45bf244168b2aa2cfd5454764 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:32:14.698071 12474 raft_consensus.cc:2258] T 120e6e4fab59474cb2a8bb365a26be49 P 9db5cca45bf244168b2aa2cfd5454764 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:32:14.698242 12475 ts_tablet_manager.cc:1905] T 120e6e4fab59474cb2a8bb365a26be49 P bcbfef3232264a4dbfa51874f4ac555e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:32:14.698271 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 12012
I20260619 20:32:14.700412 12475 ts_tablet_manager.cc:1918] T 120e6e4fab59474cb2a8bb365a26be49 P bcbfef3232264a4dbfa51874f4ac555e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:32:14.700492 12475 log.cc:1196] T 120e6e4fab59474cb2a8bb365a26be49 P bcbfef3232264a4dbfa51874f4ac555e: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-2/wal/wals/120e6e4fab59474cb2a8bb365a26be49
I20260619 20:32:14.700575 12476 ts_tablet_manager.cc:1905] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:32:14.700733 12475 ts_tablet_manager.cc:1939] T 120e6e4fab59474cb2a8bb365a26be49 P bcbfef3232264a4dbfa51874f4ac555e: Deleting consensus metadata
I20260619 20:32:14.702118 12476 ts_tablet_manager.cc:1918] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:32:14.702184 12476 log.cc:1196] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901102012824-4429-0/minicluster-data/ts-1/wal/wals/120e6e4fab59474cb2a8bb365a26be49
I20260619 20:32:14.702387 12476 ts_tablet_manager.cc:1939] T 120e6e4fab59474cb2a8bb365a26be49 P 60525532c20a4c358974dc8e75b0596d: Deleting consensus metadata
I20260619 20:32:14.702680 11926 catalog_manager.cc:4841] TS bcbfef3232264a4dbfa51874f4ac555e (127.4.83.67:36719): tablet 120e6e4fab59474cb2a8bb365a26be49 (table test-table [id=c53ab449bda440149614ba5346ce46ea]) successfully deleted
I20260619 20:32:14.702862 11927 catalog_manager.cc:4841] TS 60525532c20a4c358974dc8e75b0596d (127.4.83.66:40003): tablet 120e6e4fab59474cb2a8bb365a26be49 (table test-table [id=c53ab449bda440149614ba5346ce46ea]) successfully deleted
W20260619 20:32:14.705361 11927 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.83.65:39085 (error 108)
I20260619 20:32:14.705539 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 12153
I20260619 20:32:14.706471 12538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:14.706251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:39085 (local address 127.0.0.1:48802)
0619 20:32:14.706355 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:14.706427 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.65:39085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":46,"threads_started":1}
W20260619 20:32:14.706593 11927 catalog_manager.cc:4584] TS 9db5cca45bf244168b2aa2cfd5454764 (127.4.83.65:39085): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 120e6e4fab59474cb2a8bb365a26be49: Network error: Client connection negotiation failed: client connection to 127.4.83.65:39085: connect: Connection refused (error 111)
I20260619 20:32:14.711462 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 12293
I20260619 20:32:14.717073 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 11908
2026-06-19T20:32:14Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2775 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:32:14 dist-test-slave-4m58 krb5kdc[12541](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:32:14 dist-test-slave-4m58 krb5kdc[12541](info): set up 2 sockets
Jun 19 20:32:14 dist-test-slave-4m58 krb5kdc[12541](info): commencing operation
krb5kdc: starting...
W20260619 20:32:15.697157 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.960s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:32:15 dist-test-slave-4m58 krb5kdc[12541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:32:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:32:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
I20260619 20:32:15.814796 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:37631
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:36627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:37631
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:15.895617 12554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:15.895953 12554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:15.895991 12554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:15.897248 12554 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:32:15.897292 12554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:15.897312 12554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:15.897328 12554 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:32:15.897344 12554 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:32:15.898922 12554 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:36627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:37631
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:37631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:15.899174 12554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:15.899379 12554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:15.903944 12559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:15.903944 12562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:15.904176 12560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:15.904232 12554 server_base.cc:1034] running on GCE node
I20260619 20:32:15.904445 12554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:15.904690 12554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:15.905844 12554 hybrid_clock.cc:648] HybridClock initialized: now 1781901135905829 us; error 30 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-19T20:32:15Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1195 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:32:15 dist-test-slave-4m58 krb5kdc[12567](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:32:15 dist-test-slave-4m58 krb5kdc[12567](info): set up 2 sockets
Jun 19 20:32:15 dist-test-slave-4m58 krb5kdc[12567](info): commencing operation
krb5kdc: starting...
W20260619 20:32:16.881125 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.948s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:32:16 dist-test-slave-4m58 krb5kdc[12567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:32:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:32:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
I20260619 20:32:16.999140 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:42463
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:35547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:42463
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:17.084703 12580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:17.084935 12580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:17.084960 12580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:17.086167 12580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:32:17.086220 12580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:17.086234 12580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:17.086247 12580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:32:17.086257 12580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:32:17.087786 12580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:35547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:42463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:42463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:17.088057 12580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:17.088258 12580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:17.093156 12586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:17.093149 12585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:17.093151 12588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:17.093359 12580 server_base.cc:1034] running on GCE node
I20260619 20:32:17.093763 12580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:17.094056 12580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:17.095234 12580 hybrid_clock.cc:648] HybridClock initialized: now 1781901137095221 us; error 33 us; skew 500 ppm
Jun 19 20:32:17 dist-test-slave-4m58 krb5kdc[12567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901137, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:17.097852 12580 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:32:17.098294 12580 webserver.cc:460] Webserver started at http://127.4.83.126:36535/ using document root <none> and password file <none>
I20260619 20:32:17.098507 12580 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:17.098556 12580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:17.098685 12580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:17.099740 12580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "91191e368684436ba660fd182f7cf3d9"
format_stamp: "Formatted at 2026-06-19 20:32:17 on dist-test-slave-4m58"
server_key: "56e1e650d75e300734be9a7b181dbdc8"
server_key_iv: "4cbbe3e094cffabe553a250ff5aaf7fd"
server_key_version: "encryptionkey@0"
I20260619 20:32:17.100144 12580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "91191e368684436ba660fd182f7cf3d9"
format_stamp: "Formatted at 2026-06-19 20:32:17 on dist-test-slave-4m58"
server_key: "56e1e650d75e300734be9a7b181dbdc8"
server_key_iv: "4cbbe3e094cffabe553a250ff5aaf7fd"
server_key_version: "encryptionkey@0"
I20260619 20:32:17.101486 12580 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:17.102348 12595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:17.102552 12580 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:17.102643 12580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "91191e368684436ba660fd182f7cf3d9"
format_stamp: "Formatted at 2026-06-19 20:32:17 on dist-test-slave-4m58"
server_key: "56e1e650d75e300734be9a7b181dbdc8"
server_key_iv: "4cbbe3e094cffabe553a250ff5aaf7fd"
server_key_version: "encryptionkey@0"
I20260619 20:32:17.102716 12580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:17.136065 12580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:17.138360 12580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:17.138602 12580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:17.142688 12580 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:42463
I20260619 20:32:17.142741 12647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:42463 every 8 connection(s)
I20260619 20:32:17.143090 12580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:32:17.144078 12648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:17.144380 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 12580
I20260619 20:32:17.144486 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:32:17.144675 4429 external_mini_cluster.cc:1428] Setting key 7ccbcc7afd741a2d1e94b051323797e2
I20260619 20:32:17.146951 12648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9: Bootstrap starting.
I20260619 20:32:17.147552 12648 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:17.147806 12648 log.cc:824] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:17.148497 12648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9: No bootstrap required, opened a new log
I20260619 20:32:17.150806 12648 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91191e368684436ba660fd182f7cf3d9" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 42463 } }
I20260619 20:32:17.150976 12648 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:17.150998 12648 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91191e368684436ba660fd182f7cf3d9, State: Initialized, Role: FOLLOWER
I20260619 20:32:17.151242 12648 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91191e368684436ba660fd182f7cf3d9" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 42463 } }
I20260619 20:32:17.151355 12648 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:32:17.151389 12648 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:32:17.151444 12648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [term 0 FOLLOWER]: Advancing to term 1
Jun 19 20:32:17 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:32:17.152298 12648 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91191e368684436ba660fd182f7cf3d9" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 42463 } }
I20260619 20:32:17.152424 12648 leader_election.cc:302] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91191e368684436ba660fd182f7cf3d9; no voters:
I20260619 20:32:17.152630 12648 leader_election.cc:288] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:32:17.152781 12653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:32:17.152976 12648 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:32:17.153090 12653 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [term 1 LEADER]: Becoming Leader. State: Replica: 91191e368684436ba660fd182f7cf3d9, State: Running, Role: LEADER
I20260619 20:32:17.153232 12653 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91191e368684436ba660fd182f7cf3d9" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 42463 } }
I20260619 20:32:17.153637 12653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91191e368684436ba660fd182f7cf3d9. Latest consensus state: current_term: 1 leader_uuid: "91191e368684436ba660fd182f7cf3d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91191e368684436ba660fd182f7cf3d9" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 42463 } } }
I20260619 20:32:17.153712 12653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:17.154165 12654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91191e368684436ba660fd182f7cf3d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91191e368684436ba660fd182f7cf3d9" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 42463 } } }
I20260619 20:32:17.154331 12654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:17.153918 12662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:32:17.155205 12662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:32:17.156597 12651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:17.145306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33704 (local address 127.4.83.126:42463)
0619 20:32:17.145493 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:17.145501 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:17.145529 (+ 28us) server_negotiation.cc:407] Connection header received
0619 20:32:17.145941 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:17.145947 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:17.145992 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:17.146127 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:17.146755 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:17.147690 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:17.148641 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:17.149355 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:17.149483 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:17.152364 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:17.152369 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:17.152369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:17.152385 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:17.154617 (+ 2232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:17.155068 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:17.155071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:17.155071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:17.155166 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:17.155368 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:17.155369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:17.155370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:17.155919 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:17.156086 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:17.156237 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 20:32:17.156384 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":90,"threads_started":1}
I20260619 20:32:17.160305 12662 catalog_manager.cc:1347] Generated new cluster ID: 8d29ad8ffeef4dc48d261483932efc4c
I20260619 20:32:17.160387 12662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:32:17.186668 12662 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:32:17.187583 12662 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:32:17.201608 12662 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9: Generated new TSK 0
I20260619 20:32:17.201856 12662 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.83.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
I20260619 20:32:17.231307 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:0
--local_ip_for_outbound_sockets=127.4.83.65
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:42463
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:35547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:17.316805 12674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:17.317030 12674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:17.317056 12674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:17.318288 12674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:17.318336 12674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:17.318365 12674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:32:17.319964 12674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:35547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:42463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:17.320194 12674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:17.320387 12674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:17.325776 12680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:17.325781 12679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:17.325776 12682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:17.325960 12674 server_base.cc:1034] running on GCE node
I20260619 20:32:17.326330 12674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:17.326606 12674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:17.327777 12674 hybrid_clock.cc:648] HybridClock initialized: now 1781901137327763 us; error 27 us; skew 500 ppm
Jun 19 20:32:17 dist-test-slave-4m58 krb5kdc[12567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901137, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:17.330363 12674 init.cc:377] Logged in from keytab as kudu/127.4.83.65@KRBTEST.COM (short username kudu)
I20260619 20:32:17.330863 12674 webserver.cc:460] Webserver started at http://127.4.83.65:35403/ using document root <none> and password file <none>
I20260619 20:32:17.331071 12674 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:17.331115 12674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:17.331234 12674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:17.332351 12674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/data/instance:
uuid: "2cf0fd4743c54dd8b47c915f67a6c6d2"
format_stamp: "Formatted at 2026-06-19 20:32:17 on dist-test-slave-4m58"
server_key: "67381a36bbc00845cda655bb5731f8c2"
server_key_iv: "1400480c9d3d5baf16d0187249bbee44"
server_key_version: "encryptionkey@0"
I20260619 20:32:17.332691 12674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance:
uuid: "2cf0fd4743c54dd8b47c915f67a6c6d2"
format_stamp: "Formatted at 2026-06-19 20:32:17 on dist-test-slave-4m58"
server_key: "67381a36bbc00845cda655bb5731f8c2"
server_key_iv: "1400480c9d3d5baf16d0187249bbee44"
server_key_version: "encryptionkey@0"
I20260619 20:32:17.334039 12674 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:17.334836 12689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:17.335001 12674 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:17.335084 12674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "2cf0fd4743c54dd8b47c915f67a6c6d2"
format_stamp: "Formatted at 2026-06-19 20:32:17 on dist-test-slave-4m58"
server_key: "67381a36bbc00845cda655bb5731f8c2"
server_key_iv: "1400480c9d3d5baf16d0187249bbee44"
server_key_version: "encryptionkey@0"
I20260619 20:32:17.335155 12674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:17.359050 12674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:17.361294 12674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:17.361452 12674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:17.361691 12674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:17.361997 12674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:17.362031 12674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:17.362071 12674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:17.362094 12674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:17.368211 12674 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:41441
I20260619 20:32:17.368286 12802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:41441 every 8 connection(s)
I20260619 20:32:17.368608 12674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
Jun 19 20:32:17 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901137, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:32:17.377252 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 12674
I20260619 20:32:17.377368 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance
I20260619 20:32:17.377545 4429 external_mini_cluster.cc:1428] Setting key 4d12301c91ea226fe78c7f917d1bd2e8
I20260619 20:32:17.378026 12651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:17.369485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:45195 (local address 127.4.83.126:42463)
0619 20:32:17.369546 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:17.369548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:17.369683 (+ 135us) server_negotiation.cc:407] Connection header received
0619 20:32:17.370211 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:17.370212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:17.370225 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:17.370275 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:17.371291 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:17.371802 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:17.372800 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:17.373011 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:17.373074 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:17.375273 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:17.375274 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:17.375275 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:17.375284 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:17.376626 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:17.377008 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:17.377010 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:17.377010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:17.377088 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:17.377325 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:17.377325 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:17.377326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:17.377464 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:17.377521 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:17.377847 (+ 326us) server_negotiation.cc:299] Negotiation successful
0619 20:32:17.377941 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 20:32:17.378203 12805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:17.369344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.65:45195)
0619 20:32:17.369552 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:17.369602 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:17.370114 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:17.370343 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:17.370347 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:17.370974 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:17.371187 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:17.371191 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:17.371912 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:17.371913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:17.372715 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:17.372717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:17.373092 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:17.373092 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:17.373220 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:17.373778 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:17.373786 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:17.375179 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:17.376705 (+ 1526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:17.376714 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:17.376717 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:17.376940 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:17.377147 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:17.377148 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:17.377148 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:17.377268 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:17.377519 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:17.377526 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:17.377695 (+ 169us) client_negotiation.cc:769] Sending connection context
0619 20:32:17.377805 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 20:32:17.377973 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":84,"threads_started":1}
I20260619 20:32:17.378647 12803 heartbeater.cc:347] Connected to a master server at 127.4.83.126:42463
I20260619 20:32:17.378752 12803 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:17.379022 12803 heartbeater.cc:511] Master 127.4.83.126:42463 requested a full tablet report, sending...
I20260619 20:32:17.379585 12612 ts_manager.cc:194] Registered new tserver with Master: 2cf0fd4743c54dd8b47c915f67a6c6d2 (127.4.83.65:41441)
I20260619 20:32:17.380314 12612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.65@KRBTEST.COM'} at 127.4.83.65:45195
WARNING: no policy specified for kudu/127.4.83.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
I20260619 20:32:17.403676 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:0
--local_ip_for_outbound_sockets=127.4.83.66
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:42463
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:35547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:17.487142 12808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:17.487389 12808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:17.487416 12808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:17.488794 12808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:17.488850 12808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:17.488881 12808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:32:17.490483 12808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:35547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:42463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:17.490741 12808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:17.490959 12808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:17.496881 12814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:17.497095 12808 server_base.cc:1034] running on GCE node
W20260619 20:32:17.496869 12813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:17.496869 12816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:17.497454 12808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:17.497754 12808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:17.498960 12808 hybrid_clock.cc:648] HybridClock initialized: now 1781901137498948 us; error 47 us; skew 500 ppm
Jun 19 20:32:17 dist-test-slave-4m58 krb5kdc[12567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901137, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:17.502024 12808 init.cc:377] Logged in from keytab as kudu/127.4.83.66@KRBTEST.COM (short username kudu)
I20260619 20:32:17.502524 12808 webserver.cc:460] Webserver started at http://127.4.83.66:46195/ using document root <none> and password file <none>
I20260619 20:32:17.502744 12808 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:17.502789 12808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:17.502900 12808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:17.504046 12808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/data/instance:
uuid: "45923f706ffd4ef08c9048fb7943313b"
format_stamp: "Formatted at 2026-06-19 20:32:17 on dist-test-slave-4m58"
server_key: "bee21bec54eeb877d938ca63a4567a19"
server_key_iv: "52302a8c3e1f99f6faf46a4ba425e0b8"
server_key_version: "encryptionkey@0"
I20260619 20:32:17.504360 12808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance:
uuid: "45923f706ffd4ef08c9048fb7943313b"
format_stamp: "Formatted at 2026-06-19 20:32:17 on dist-test-slave-4m58"
server_key: "bee21bec54eeb877d938ca63a4567a19"
server_key_iv: "52302a8c3e1f99f6faf46a4ba425e0b8"
server_key_version: "encryptionkey@0"
I20260619 20:32:17.505681 12808 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:17.506466 12823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:17.506680 12808 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:17.506788 12808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "45923f706ffd4ef08c9048fb7943313b"
format_stamp: "Formatted at 2026-06-19 20:32:17 on dist-test-slave-4m58"
server_key: "bee21bec54eeb877d938ca63a4567a19"
server_key_iv: "52302a8c3e1f99f6faf46a4ba425e0b8"
server_key_version: "encryptionkey@0"
I20260619 20:32:17.506865 12808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:17.529094 12808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:17.531263 12808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:17.531405 12808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:17.531682 12808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:17.532037 12808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:17.532074 12808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:17.532105 12808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:17.532130 12808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:17.537806 12808 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:44615
I20260619 20:32:17.537858 12936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:44615 every 8 connection(s)
I20260619 20:32:17.538187 12808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
I20260619 20:32:17.539060 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 12808
I20260619 20:32:17.539175 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance
I20260619 20:32:17.539376 4429 external_mini_cluster.cc:1428] Setting key 94c831c67ec4925df312e0498e7c5033
Jun 19 20:32:17 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901137, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.83.67@KRBTEST.COM; defaulting to no policy
I20260619 20:32:17.548640 12651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:17.538925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:51657 (local address 127.4.83.126:42463)
0619 20:32:17.538994 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:17.538996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:17.539255 (+ 259us) server_negotiation.cc:407] Connection header received
0619 20:32:17.539762 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:17.539763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:17.539773 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:17.539824 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:17.540785 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:17.541459 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:17.542874 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:17.543091 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:17.543157 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:17.545726 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:17.545728 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:17.545728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:17.545736 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:17.547116 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:17.547595 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:17.547596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:17.547597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:17.547649 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:17.547980 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:17.547981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:17.547981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:17.548116 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:17.548181 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:17.548481 (+ 300us) server_negotiation.cc:299] Negotiation successful
0619 20:32:17.548567 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:32:17.548768 12939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:17.538919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.66:51657)
0619 20:32:17.539102 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:17.539149 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:17.539664 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:17.539923 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:17.539927 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:17.540499 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:17.540689 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:17.540694 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:17.541551 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:17.541552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:17.542752 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:17.542757 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:17.543181 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:17.543182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:17.543336 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:17.543935 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:17.543945 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:17.545608 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:17.547202 (+ 1594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:17.547211 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:17.547213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:17.547524 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:17.547712 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:17.547713 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:17.547713 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:17.547821 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:17.548206 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:17.548212 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:17.548356 (+ 144us) client_negotiation.cc:769] Sending connection context
0619 20:32:17.548438 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 20:32:17.548580 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":107,"threads_started":1}
I20260619 20:32:17.549285 12937 heartbeater.cc:347] Connected to a master server at 127.4.83.126:42463
I20260619 20:32:17.549391 12937 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:17.549590 12937 heartbeater.cc:511] Master 127.4.83.126:42463 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.67@KRBTEST.COM" created.
I20260619 20:32:17.549997 12612 ts_manager.cc:194] Registered new tserver with Master: 45923f706ffd4ef08c9048fb7943313b (127.4.83.66:44615)
I20260619 20:32:17.550391 12612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.66@KRBTEST.COM'} at 127.4.83.66:51657
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
I20260619 20:32:17.564549 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:0
--local_ip_for_outbound_sockets=127.4.83.67
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:42463
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:35547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:17.647599 12942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:17.647810 12942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:17.647873 12942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:17.649178 12942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:17.649224 12942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:17.649252 12942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:32:17.650782 12942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:35547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:42463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:17.651024 12942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:17.651245 12942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:17.656853 12948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:17.656985 12942 server_base.cc:1034] running on GCE node
W20260619 20:32:17.656863 12950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:17.656850 12947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:17.657280 12942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:17.657507 12942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:17.658682 12942 hybrid_clock.cc:648] HybridClock initialized: now 1781901137658670 us; error 29 us; skew 500 ppm
Jun 19 20:32:17 dist-test-slave-4m58 krb5kdc[12567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901137, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:17.661154 12942 init.cc:377] Logged in from keytab as kudu/127.4.83.67@KRBTEST.COM (short username kudu)
I20260619 20:32:17.661594 12942 webserver.cc:460] Webserver started at http://127.4.83.67:42603/ using document root <none> and password file <none>
I20260619 20:32:17.661808 12942 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:17.661854 12942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:17.661975 12942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:17.662995 12942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/data/instance:
uuid: "d4a19de23b054993af19d31c4f943353"
format_stamp: "Formatted at 2026-06-19 20:32:17 on dist-test-slave-4m58"
server_key: "517dc22b1acb6ed295f8ae0dab50d486"
server_key_iv: "1620d079bf73a6f09fb21143e62e46ca"
server_key_version: "encryptionkey@0"
I20260619 20:32:17.663314 12942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance:
uuid: "d4a19de23b054993af19d31c4f943353"
format_stamp: "Formatted at 2026-06-19 20:32:17 on dist-test-slave-4m58"
server_key: "517dc22b1acb6ed295f8ae0dab50d486"
server_key_iv: "1620d079bf73a6f09fb21143e62e46ca"
server_key_version: "encryptionkey@0"
I20260619 20:32:17.664669 12942 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:32:17.665472 12957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:17.665663 12942 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:17.665752 12942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "d4a19de23b054993af19d31c4f943353"
format_stamp: "Formatted at 2026-06-19 20:32:17 on dist-test-slave-4m58"
server_key: "517dc22b1acb6ed295f8ae0dab50d486"
server_key_iv: "1620d079bf73a6f09fb21143e62e46ca"
server_key_version: "encryptionkey@0"
I20260619 20:32:17.665817 12942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:17.688297 12942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:17.690490 12942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:17.690636 12942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:17.690862 12942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:17.691160 12942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:17.691192 12942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:17.691222 12942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:17.691244 12942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:17.696707 12942 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:44131
I20260619 20:32:17.696753 13070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:44131 every 8 connection(s)
I20260619 20:32:17.697072 12942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
I20260619 20:32:17.699870 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 12942
I20260619 20:32:17.699960 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance
I20260619 20:32:17.700115 4429 external_mini_cluster.cc:1428] Setting key 7b57e80130e144f8bfd28427817afeac
Jun 19 20:32:17 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901137, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:32:17.706429 12651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:17.697781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:52757 (local address 127.4.83.126:42463)
0619 20:32:17.697839 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:17.697841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:17.698037 (+ 196us) server_negotiation.cc:407] Connection header received
0619 20:32:17.698529 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:17.698530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:17.698543 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:17.698592 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:17.699535 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:17.700120 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:17.701379 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:17.701633 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:17.701727 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:17.703917 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:17.703919 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:17.703919 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:17.703926 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:17.705219 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:17.705594 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:17.705595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:17.705595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:17.705673 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:17.705846 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:17.705846 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:17.705846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:17.705957 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:17.706015 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:17.706297 (+ 282us) server_negotiation.cc:299] Negotiation successful
0619 20:32:17.706361 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 20:32:17.706583 13073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:17.697759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.67:52757)
0619 20:32:17.697913 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:17.697956 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:17.698437 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:17.698669 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:17.698673 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:17.699235 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:17.699440 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:17.699444 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:17.700237 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:17.700239 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:17.701283 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:17.701285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:17.701733 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:17.701734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:17.701871 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:17.702397 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:17.702405 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:17.703806 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:17.705290 (+ 1484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:17.705296 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:17.705298 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:17.705535 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:17.705718 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:17.705719 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:17.705719 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:17.705810 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:17.706046 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:17.706061 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:17.706187 (+ 126us) client_negotiation.cc:769] Sending connection context
0619 20:32:17.706272 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 20:32:17.706395 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":74,"threads_started":1}
I20260619 20:32:17.707007 13071 heartbeater.cc:347] Connected to a master server at 127.4.83.126:42463
I20260619 20:32:17.707104 13071 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:17.707253 13071 heartbeater.cc:511] Master 127.4.83.126:42463 requested a full tablet report, sending...
I20260619 20:32:17.707587 12612 ts_manager.cc:194] Registered new tserver with Master: d4a19de23b054993af19d31c4f943353 (127.4.83.67:44131)
I20260619 20:32:17.707968 12612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.67@KRBTEST.COM'} at 127.4.83.67:52757
I20260619 20:32:17.711875 4429 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.4.83.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.83.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.83.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.65.keytab.
Entry for principal oryx/127.4.83.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.65.keytab.
WARNING: no policy specified for oryx/127.4.83.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.83.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.83.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.66.keytab.
Entry for principal oryx/127.4.83.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.66.keytab.
WARNING: no policy specified for oryx/127.4.83.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.83.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.83.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.67.keytab.
Entry for principal oryx/127.4.83.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.67.keytab.
I20260619 20:32:17.790896 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 12674
I20260619 20:32:17.794996 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 12808
I20260619 20:32:17.798575 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 12942
I20260619 20:32:17.802347 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 12580
I20260619 20:32:17.806835 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:42463
--webserver_interface=127.4.83.126
--webserver_port=36535
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:35547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:42463
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:17.888047 13080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:17.888388 13080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:17.888422 13080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:17.891455 13080 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:32:17.891551 13080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:17.891569 13080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:17.891583 13080 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:32:17.891604 13080 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:32:17.895221 13080 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:35547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:42463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:42463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=36535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:17.895551 13080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:17.895769 13080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:17.902666 13085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:17.902671 13088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:17.902671 13086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:17.903215 13080 server_base.cc:1034] running on GCE node
I20260619 20:32:17.903563 13080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:17.904042 13080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:17.905256 13080 hybrid_clock.cc:648] HybridClock initialized: now 1781901137905238 us; error 34 us; skew 500 ppm
Jun 19 20:32:17 dist-test-slave-4m58 krb5kdc[12567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901137, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:17.909126 13080 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:32:17.909786 13080 webserver.cc:460] Webserver started at http://127.4.83.126:36535/ using document root <none> and password file <none>
I20260619 20:32:17.910087 13080 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:17.910171 13080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:17.912178 13080 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260619 20:32:17.913077 13095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:17.913321 13080 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:17.913429 13080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "91191e368684436ba660fd182f7cf3d9"
format_stamp: "Formatted at 2026-06-19 20:32:17 on dist-test-slave-4m58"
server_key: "56e1e650d75e300734be9a7b181dbdc8"
server_key_iv: "4cbbe3e094cffabe553a250ff5aaf7fd"
server_key_version: "encryptionkey@0"
I20260619 20:32:17.913925 13080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:17.931398 13080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:17.934329 13080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:17.934545 13080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:17.939471 13080 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:42463
I20260619 20:32:17.939558 13147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:42463 every 8 connection(s)
I20260619 20:32:17.940063 13080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:32:17.941833 13148 sys_catalog.cc:263] Verifying existing consensus state
I20260619 20:32:17.942324 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 13080
I20260619 20:32:17.942602 13148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9: Bootstrap starting.
I20260619 20:32:17.942654 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:41441
--local_ip_for_outbound_sockets=127.4.83.65
--tserver_master_addrs=127.4.83.126:42463
--webserver_port=35403
--webserver_interface=127.4.83.65
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:35547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.83.65
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:32:17.952553 13148 log.cc:824] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:17.954293 13148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:32:17.954609 13148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9: Bootstrap complete.
I20260619 20:32:17.957818 13148 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91191e368684436ba660fd182f7cf3d9" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 42463 } }
I20260619 20:32:17.958122 13148 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91191e368684436ba660fd182f7cf3d9, State: Initialized, Role: FOLLOWER
I20260619 20:32:17.958294 13148 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91191e368684436ba660fd182f7cf3d9" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 42463 } }
I20260619 20:32:17.958374 13148 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:32:17.958425 13148 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:32:17.958472 13148 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [term 1 FOLLOWER]: Advancing to term 2
I20260619 20:32:17.959607 13148 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91191e368684436ba660fd182f7cf3d9" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 42463 } }
I20260619 20:32:17.959720 13148 leader_election.cc:302] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91191e368684436ba660fd182f7cf3d9; no voters:
I20260619 20:32:17.959980 13148 leader_election.cc:288] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 20:32:17.960032 13152 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 20:32:17.960175 13152 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [term 2 LEADER]: Becoming Leader. State: Replica: 91191e368684436ba660fd182f7cf3d9, State: Running, Role: LEADER
I20260619 20:32:17.960317 13148 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:32:17.960438 13152 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91191e368684436ba660fd182f7cf3d9" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 42463 } }
I20260619 20:32:17.960898 13153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "91191e368684436ba660fd182f7cf3d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91191e368684436ba660fd182f7cf3d9" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 42463 } } }
I20260619 20:32:17.961022 13153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:17.960917 13154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91191e368684436ba660fd182f7cf3d9. Latest consensus state: current_term: 2 leader_uuid: "91191e368684436ba660fd182f7cf3d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91191e368684436ba660fd182f7cf3d9" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 42463 } } }
I20260619 20:32:17.961098 13154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:17.961705 13161 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:32:17.962286 13161 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:32:17.962723 13161 catalog_manager.cc:1259] Loaded cluster ID: 8d29ad8ffeef4dc48d261483932efc4c
I20260619 20:32:17.962808 13161 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:32:17.964818 13161 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:32:17.965269 13161 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 91191e368684436ba660fd182f7cf3d9: Loaded TSK: 0
I20260619 20:32:17.965574 13161 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 20:32:18.042624 13150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:18.042946 13150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:18.042979 13150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:18.044910 13150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:18.045028 13150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:18.045069 13150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:32:18.047451 13150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:35547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.65.keytab
--principal=oryx/127.4.83.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:41441
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=35403
--tserver_master_addrs=127.4.83.126:42463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:18.047920 13150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:18.048235 13150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:18.054244 13174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:18.054257 13175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:18.054555 13150 server_base.cc:1034] running on GCE node
W20260619 20:32:18.054328 13177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:18.054998 13150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:18.055326 13150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:18.056541 13150 hybrid_clock.cc:648] HybridClock initialized: now 1781901138056506 us; error 54 us; skew 500 ppm
Jun 19 20:32:18 dist-test-slave-4m58 krb5kdc[12567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901138, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:18.060415 13150 init.cc:377] Logged in from keytab as oryx/127.4.83.65@KRBTEST.COM (short username oryx)
I20260619 20:32:18.061182 13150 webserver.cc:460] Webserver started at http://127.4.83.65:35403/ using document root <none> and password file <none>
I20260619 20:32:18.061475 13150 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:18.061544 13150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:18.063344 13150 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:18.064217 13184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:18.064515 13150 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:18.064651 13150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "2cf0fd4743c54dd8b47c915f67a6c6d2"
format_stamp: "Formatted at 2026-06-19 20:32:17 on dist-test-slave-4m58"
server_key: "67381a36bbc00845cda655bb5731f8c2"
server_key_iv: "1400480c9d3d5baf16d0187249bbee44"
server_key_version: "encryptionkey@0"
I20260619 20:32:18.065155 13150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:18.079757 13150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:18.082804 13150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:18.082988 13150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:18.083323 13150 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:18.083783 13150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:18.083865 13150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:18.083920 13150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:18.083950 13150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:18.093029 13150 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:41441
I20260619 20:32:18.093134 13297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:41441 every 8 connection(s)
I20260619 20:32:18.093562 13150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
I20260619 20:32:18.099910 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 13150
I20260619 20:32:18.100208 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:44615
--local_ip_for_outbound_sockets=127.4.83.66
--tserver_master_addrs=127.4.83.126:42463
--webserver_port=46195
--webserver_interface=127.4.83.66
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:35547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.83.66
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 20:32:18 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:18 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:18.103356 13300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:18.094441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.65:54991)
0619 20:32:18.094709 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:18.094769 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:18.095379 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:18.095797 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:18.095800 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:18.096475 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:18.096685 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:18.096690 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.097807 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.097808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:18.098641 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:18.098644 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.099232 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.099233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:18.099364 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:18.100048 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:18.100064 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:18.103134 (+ 3070us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":133,"threads_started":1}
W20260619 20:32:18.103435 13301 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:18.094564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:54991 (local address 127.4.83.126:42463)
0619 20:32:18.094802 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:18.094810 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:18.094912 (+ 102us) server_negotiation.cc:407] Connection header received
0619 20:32:18.095592 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:18.095597 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:18.095638 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:18.095753 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:18.096807 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.097708 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.098735 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.099148 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.099245 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:18.103291 (+ 4046us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.65:54991: BlockingRecv error: recv got EOF from 127.4.83.65:54991 (error 108)
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":100,"threads_started":1}
W20260619 20:32:18.103628 13298 heartbeater.cc:644] Failed to heartbeat to 127.4.83.126:42463 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.83.126:42463: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Jun 19 20:32:18 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:18 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:18.109112 13303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:18.103865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.65:36771)
0619 20:32:18.104021 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:18.104032 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:18.104098 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:18.104231 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:18.104232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:18.104401 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:18.104449 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:18.104450 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.105072 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.105073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:18.105607 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:18.105608 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.105949 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.105950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:18.106007 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:18.106441 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:18.106446 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:18.109016 (+ 2570us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":105,"threads_started":1}
W20260619 20:32:18.109184 13301 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:18.103911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:36771 (local address 127.4.83.126:42463)
0619 20:32:18.103961 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:18.103963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:18.104098 (+ 135us) server_negotiation.cc:407] Connection header received
0619 20:32:18.104159 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:18.104159 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:18.104173 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:18.104222 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:18.104525 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.105004 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.105676 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.105870 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.105925 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:18.109130 (+ 3205us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.65:36771: BlockingRecv error: recv got EOF from 127.4.83.65:36771 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Jun 19 20:32:18 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:18 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:18.114866 13303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:18.109412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.65:45719)
0619 20:32:18.109443 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:18.109463 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:18.109721 (+ 258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:18.109953 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:18.109955 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:18.110123 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:18.110180 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:18.110182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.111198 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.111199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:18.111746 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:18.111748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.112298 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.112299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:18.112358 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:18.112775 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:18.112780 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:18.114810 (+ 2030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":15}
W20260619 20:32:18.114998 13298 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 20:32:18.115067 13301 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:18.109511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:45719 (local address 127.4.83.126:42463)
0619 20:32:18.109607 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:18.109609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:18.109734 (+ 125us) server_negotiation.cc:407] Connection header received
0619 20:32:18.109788 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:18.109790 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:18.109811 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:18.109917 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:18.110268 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.111067 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.111877 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.112210 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.112303 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:18.114974 (+ 2671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.65:45719: BlockingRecv error: recv got EOF from 127.4.83.65:45719 (error 108)
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":127,"threads_started":1}
W20260619 20:32:18.197799 13302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:18.198077 13302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:18.198110 13302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:18.200507 13302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:18.200620 13302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:18.200660 13302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:32:18.202867 13302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:35547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.66.keytab
--principal=oryx/127.4.83.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:44615
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=46195
--tserver_master_addrs=127.4.83.126:42463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:18.203197 13302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:18.203460 13302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:18.209399 13312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:18.209493 13310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:18.209493 13309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:18.209765 13302 server_base.cc:1034] running on GCE node
I20260619 20:32:18.210116 13302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:18.210494 13302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:18.211681 13302 hybrid_clock.cc:648] HybridClock initialized: now 1781901138211661 us; error 44 us; skew 500 ppm
Jun 19 20:32:18 dist-test-slave-4m58 krb5kdc[12567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901138, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:18.214866 13302 init.cc:377] Logged in from keytab as oryx/127.4.83.66@KRBTEST.COM (short username oryx)
I20260619 20:32:18.215516 13302 webserver.cc:460] Webserver started at http://127.4.83.66:46195/ using document root <none> and password file <none>
I20260619 20:32:18.215790 13302 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:18.215925 13302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:18.217702 13302 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:18.218468 13319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:18.218743 13302 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:18.218869 13302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "45923f706ffd4ef08c9048fb7943313b"
format_stamp: "Formatted at 2026-06-19 20:32:17 on dist-test-slave-4m58"
server_key: "bee21bec54eeb877d938ca63a4567a19"
server_key_iv: "52302a8c3e1f99f6faf46a4ba425e0b8"
server_key_version: "encryptionkey@0"
I20260619 20:32:18.219385 13302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:18.234081 13302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:18.236932 13302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:18.237105 13302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:18.237413 13302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:18.237795 13302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:18.237831 13302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:18.237875 13302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:18.237898 13302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:18.245819 13302 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:44615
I20260619 20:32:18.245944 13432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:44615 every 8 connection(s)
I20260619 20:32:18.246328 13302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
I20260619 20:32:18.246763 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 13302
I20260619 20:32:18.247087 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:44131
--local_ip_for_outbound_sockets=127.4.83.67
--tserver_master_addrs=127.4.83.126:42463
--webserver_port=42603
--webserver_interface=127.4.83.67
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:35547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.83.67
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 20:32:18 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:18 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
W20260619 20:32:18.255615 13301 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:18.247269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:41889 (local address 127.4.83.126:42463)
0619 20:32:18.247328 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:18.247331 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:18.247650 (+ 319us) server_negotiation.cc:407] Connection header received
0619 20:32:18.248393 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:18.248395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:18.248411 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:18.248496 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:18.249691 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.250237 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.251451 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.251662 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.251725 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:18.255548 (+ 3823us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.66:41889: BlockingRecv error: recv got EOF from 127.4.83.66:41889 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 20:32:18.255666 13435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:18.247214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.66:41889)
0619 20:32:18.247500 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:18.247549 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:18.248270 (+ 721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:18.248632 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:18.248638 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:18.249302 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:18.249564 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:18.249570 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.250332 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.250333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:18.251360 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:18.251364 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.251740 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.251741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:18.251935 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:18.252616 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:18.252627 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:18.255403 (+ 2776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":97,"threads_started":1}
W20260619 20:32:18.256120 13433 heartbeater.cc:644] Failed to heartbeat to 127.4.83.126:42463 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.83.126:42463: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Jun 19 20:32:18 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:18 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:18.262490 13435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:18.256439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.66:52911)
0619 20:32:18.256529 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:18.256543 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:18.256611 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:18.256794 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:18.256795 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:18.256969 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:18.257038 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:18.257041 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.257924 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.257926 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:18.258621 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:18.258625 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.259046 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.259047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:18.259131 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:18.259751 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:18.259756 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:18.262394 (+ 2638us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":61}
W20260619 20:32:18.262611 13301 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:18.256553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:52911 (local address 127.4.83.126:42463)
0619 20:32:18.256633 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:18.256636 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:18.256645 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:32:18.256683 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:18.256683 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:18.256695 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:18.256786 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:18.257114 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.257792 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.258745 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.258971 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.259043 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:18.262540 (+ 3497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.66:52911: BlockingRecv error: recv got EOF from 127.4.83.66:52911 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Jun 19 20:32:18 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:18 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:18.268549 13435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:18.263190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.66:39959)
0619 20:32:18.263279 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:18.263294 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:18.263363 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:18.263507 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:18.263508 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:18.263683 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:18.263738 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:18.263740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.264501 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.264502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:18.265123 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:18.265126 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.265500 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.265501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:18.265554 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:18.266094 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:18.266099 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:18.268474 (+ 2375us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":57}
W20260619 20:32:18.268738 13433 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 20:32:18.268945 13301 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:18.263300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:39959 (local address 127.4.83.126:42463)
0619 20:32:18.263371 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:18.263373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:18.263403 (+ 30us) server_negotiation.cc:407] Connection header received
0619 20:32:18.263436 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:18.263437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:18.263448 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:18.263494 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:18.263818 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.264406 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.265229 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.265435 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.265500 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:18.268869 (+ 3369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.66:39959: BlockingRecv error: recv got EOF from 127.4.83.66:39959 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260619 20:32:18.335147 13436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:18.335395 13436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:18.335429 13436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:18.336681 13436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:18.336733 13436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:18.336762 13436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:32:18.338304 13436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:35547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/krb5kdc/oryx_127.4.83.67.keytab
--principal=oryx/127.4.83.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:44131
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=42603
--tserver_master_addrs=127.4.83.126:42463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:18.338542 13436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:18.338749 13436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:18.344168 13442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:18.344185 13441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:18.344386 13436 server_base.cc:1034] running on GCE node
W20260619 20:32:18.344251 13444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:18.344708 13436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:18.345021 13436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:18.346213 13436 hybrid_clock.cc:648] HybridClock initialized: now 1781901138346152 us; error 84 us; skew 500 ppm
Jun 19 20:32:18 dist-test-slave-4m58 krb5kdc[12567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901138, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.83.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:18.349095 13436 init.cc:377] Logged in from keytab as oryx/127.4.83.67@KRBTEST.COM (short username oryx)
I20260619 20:32:18.349627 13436 webserver.cc:460] Webserver started at http://127.4.83.67:42603/ using document root <none> and password file <none>
I20260619 20:32:18.349851 13436 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:18.349913 13436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:18.351639 13436 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:32:18.352545 13451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:18.352846 13436 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:32:18.352972 13436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "d4a19de23b054993af19d31c4f943353"
format_stamp: "Formatted at 2026-06-19 20:32:17 on dist-test-slave-4m58"
server_key: "517dc22b1acb6ed295f8ae0dab50d486"
server_key_iv: "1620d079bf73a6f09fb21143e62e46ca"
server_key_version: "encryptionkey@0"
I20260619 20:32:18.353482 13436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:18.370973 13436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:18.373689 13436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:18.373850 13436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:18.374162 13436 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:18.374560 13436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:18.374598 13436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:18.374624 13436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:18.374635 13436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:18.383076 13436 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:44131
I20260619 20:32:18.383210 13564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:44131 every 8 connection(s)
I20260619 20:32:18.383598 13436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
Jun 19 20:32:18 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:18 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:18.392328 13567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:18.384584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.67:33071)
0619 20:32:18.384777 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:18.384829 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:18.385419 (+ 590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:18.385711 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:18.385715 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:18.386364 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:18.386615 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:18.386619 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.387338 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.387338 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:18.388352 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:18.388356 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.388813 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.388814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:18.388943 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:18.389561 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:18.389570 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:18.392115 (+ 2545us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":91,"threads_started":1}
W20260619 20:32:18.392318 13301 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:18.384605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:33071 (local address 127.4.83.126:42463)
0619 20:32:18.384683 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:18.384685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:18.384962 (+ 277us) server_negotiation.cc:407] Connection header received
0619 20:32:18.385540 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:18.385542 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:18.385553 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:18.385641 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:18.386717 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.387228 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.388509 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.388715 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.388790 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:18.392258 (+ 3468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.67:33071: BlockingRecv error: recv got EOF from 127.4.83.67:33071 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260619 20:32:18.392673 13565 heartbeater.cc:644] Failed to heartbeat to 127.4.83.126:42463 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.83.126:42463: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
I20260619 20:32:18.393525 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 13436
Jun 19 20:32:18 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:18 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:18.398355 13567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:18.392940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.67:48505)
0619 20:32:18.393022 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:18.393038 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:18.393100 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:18.393341 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:18.393343 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:18.393482 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:18.393547 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:18.393549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.394384 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.394398 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:18.395001 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:18.395004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.395369 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.395370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:18.395451 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:18.395956 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:18.395963 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:18.398274 (+ 2311us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260619 20:32:18.398540 13301 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:18.393170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:48505 (local address 127.4.83.126:42463)
0619 20:32:18.393209 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:18.393211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:18.393219 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:32:18.393242 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:18.393242 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:18.393250 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:18.393312 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:18.393649 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.394284 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.395093 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.395296 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.395350 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:18.398378 (+ 3028us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.67:48505: BlockingRecv error: recv got EOF from 127.4.83.67:48505 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:32:18.401943 13304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:18.393986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33716 (local address 127.4.83.126:42463)
0619 20:32:18.394073 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:18.394076 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:18.394292 (+ 216us) server_negotiation.cc:407] Connection header received
0619 20:32:18.394327 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:18.394328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:18.394362 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:18.394471 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:18.395064 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.395674 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.396767 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.396990 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.397050 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:18.398367 (+ 1317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:18.398375 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:18.398375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:18.398394 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:18.400493 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:18.400862 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:18.400866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:18.400867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:18.400967 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:18.401161 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:18.401162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:18.401162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:18.401443 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:18.401603 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:18.401711 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 20:32:18.401835 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Jun 19 20:32:18 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:18 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:18.403947 13567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:18.398652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.67:40429)
0619 20:32:18.398820 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:18.398831 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:18.398875 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:18.399048 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:18.399049 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:18.399217 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:18.399269 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:18.399271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.400176 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.400177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:18.400773 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:18.400775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.401173 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.401174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:18.401226 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:18.401598 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:18.401604 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:18.403815 (+ 2211us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152}
W20260619 20:32:18.404004 13301 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:18.398796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:40429 (local address 127.4.83.126:42463)
0619 20:32:18.398864 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:18.398866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:18.398875 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:32:18.398949 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:18.398950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:18.398961 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:18.399023 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:18.399354 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.400070 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.400894 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:18.401106 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:18.401170 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:18.403945 (+ 2775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.67:40429: BlockingRecv error: recv got EOF from 127.4.83.67:40429 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260619 20:32:18.404179 13565 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 19 20:32:19 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:19 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:19.125777 13569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:19.115678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.65:46981)
0619 20:32:19.115877 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:19.115890 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:19.115984 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:19.116161 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:19.116162 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:19.116376 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:19.116439 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:19.116441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:19.117692 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:19.117693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:19.118378 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:19.118383 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:19.118783 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:19.118784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:19.118847 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:19.122961 (+ 4114us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:19.122971 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:19.125635 (+ 2664us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":87,"threads_started":1}
W20260619 20:32:19.125877 13570 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:19.115768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:46981 (local address 127.4.83.126:42463)
0619 20:32:19.115929 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:19.115931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:19.115958 (+ 27us) server_negotiation.cc:407] Connection header received
0619 20:32:19.116047 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:19.116048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:19.116067 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:19.116137 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:19.116512 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:19.117425 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:19.118476 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:19.118709 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:19.118775 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:19.125802 (+ 7027us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.65:46981: BlockingRecv error: recv got EOF from 127.4.83.65:46981 (error 108)
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":91,"threads_started":1}
Jun 19 20:32:19 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:19 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:19.275576 13571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:19.269348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.66:51923)
0619 20:32:19.269617 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:19.269634 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:19.269725 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:19.269936 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:19.269938 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:19.270177 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:19.270249 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:19.270252 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:19.271082 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:19.271083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:19.271689 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:19.271691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:19.272235 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:19.272237 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:19.272314 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:19.273199 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:19.273206 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:19.275462 (+ 2256us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":209,"threads_started":1}
W20260619 20:32:19.275678 13570 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:19.269506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:51923 (local address 127.4.83.126:42463)
0619 20:32:19.269580 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:19.269582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:19.269720 (+ 138us) server_negotiation.cc:407] Connection header received
0619 20:32:19.269781 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:19.269782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:19.269795 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:19.269879 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:19.270367 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:19.270948 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:19.271888 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:19.272126 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:19.272199 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:19.275614 (+ 3415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.66:51923: BlockingRecv error: recv got EOF from 127.4.83.66:51923 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Jun 19 20:32:19 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:19 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:19.410620 13572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:19.404886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.67:60287)
0619 20:32:19.405065 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:19.405081 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:19.405183 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:19.405345 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:19.405346 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:19.405570 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:19.405633 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:19.405635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:19.406352 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:19.406353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:19.406948 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:19.406950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:19.407373 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:19.407374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:19.407448 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:19.408109 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:19.408115 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:19.410525 (+ 2410us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":85,"threads_started":1}
W20260619 20:32:19.410750 13570 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:19.405040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:60287 (local address 127.4.83.126:42463)
0619 20:32:19.405102 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:19.405105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:19.405163 (+ 58us) server_negotiation.cc:407] Connection header received
0619 20:32:19.405257 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:19.405258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:19.405273 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:19.405327 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:19.405712 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:19.406268 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:19.407030 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:19.407238 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:19.407316 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:19.410677 (+ 3361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.67:60287: BlockingRecv error: recv got EOF from 127.4.83.67:60287 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Jun 19 20:32:20 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:20 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:20.132297 13573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:20.126496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.65:48089)
0619 20:32:20.126682 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:20.126694 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:20.126762 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:20.127036 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:20.127037 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:20.127286 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:20.127380 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:20.127383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:20.128231 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:20.128233 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:20.128938 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:20.128941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:20.129329 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:20.129331 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:20.129395 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:20.129931 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:20.129936 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:20.132215 (+ 2279us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":92,"threads_started":1}
W20260619 20:32:20.132409 13574 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:20.126627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:48089 (local address 127.4.83.126:42463)
0619 20:32:20.126778 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:20.126780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:20.126832 (+ 52us) server_negotiation.cc:407] Connection header received
0619 20:32:20.126859 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:20.126859 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:20.126871 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:20.126938 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:20.127469 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:20.128061 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:20.129015 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:20.129225 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:20.129287 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:20.132334 (+ 3047us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.65:48089: BlockingRecv error: recv got EOF from 127.4.83.65:48089 (error 108)
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
Jun 19 20:32:20 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:20 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:20.281081 13575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:20.276194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.66:59735)
0619 20:32:20.276315 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:20.276327 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:20.276382 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:20.276562 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:20.276563 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:20.276743 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:20.276792 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:20.276794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:20.277434 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:20.277435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:20.277965 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:20.277966 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:20.278285 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:20.278287 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:20.278337 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:20.278938 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:20.278942 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:20.280968 (+ 2026us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":68,"threads_started":1}
W20260619 20:32:20.281127 13574 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:20.276361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:59735 (local address 127.4.83.126:42463)
0619 20:32:20.276449 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:20.276452 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:20.276458 (+ 6us) server_negotiation.cc:407] Connection header received
0619 20:32:20.276480 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:20.276480 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:20.276493 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:20.276543 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:20.276866 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:20.277360 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:20.278045 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:20.278228 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:20.278282 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:20.281077 (+ 2795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.66:59735: BlockingRecv error: recv got EOF from 127.4.83.66:59735 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 19 20:32:20 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:20 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:20.416216 13576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:20.411313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.67:40903)
0619 20:32:20.411509 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:20.411525 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:20.411600 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:20.411771 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:20.411772 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:20.411995 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:20.412049 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:20.412051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:20.412701 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:20.412702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:20.413235 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:20.413237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:20.413558 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:20.413558 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:20.413609 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:20.414113 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:20.414117 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:20.416127 (+ 2010us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":79,"threads_started":1}
W20260619 20:32:20.416312 13574 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:20.411580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:40903 (local address 127.4.83.126:42463)
0619 20:32:20.411656 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:20.411659 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:20.411666 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:32:20.411693 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:20.411694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:20.411706 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:20.411754 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:20.412130 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:20.412632 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:20.413308 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:20.413499 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:20.413553 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:20.416239 (+ 2686us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.67:40903: BlockingRecv error: recv got EOF from 127.4.83.67:40903 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Jun 19 20:32:21 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:21 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:21.139123 13577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:21.133005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.65:58821)
0619 20:32:21.133183 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:21.133213 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:21.133297 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:21.133488 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:21.133489 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:21.133703 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:21.133770 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:21.133773 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:21.134652 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:21.134653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:21.135267 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:21.135270 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:21.135739 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:21.135740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:21.135793 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:21.136473 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:21.136480 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:21.139021 (+ 2541us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":104,"threads_started":1}
W20260619 20:32:21.139266 13578 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:21.133130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:58821 (local address 127.4.83.126:42463)
0619 20:32:21.133280 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:21.133282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:21.133295 (+ 13us) server_negotiation.cc:407] Connection header received
0619 20:32:21.133366 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:21.133366 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:21.133379 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:21.133441 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:21.133881 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:21.134478 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:21.135375 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:21.135648 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:21.135724 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:21.139191 (+ 3467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.65:58821: BlockingRecv error: recv got EOF from 127.4.83.65:58821 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":77,"threads_started":1}
Jun 19 20:32:21 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:21 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:21.287448 13579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:21.281846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.66:33697)
0619 20:32:21.282053 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:21.282067 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:21.282187 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:21.282360 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:21.282361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:21.282570 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:21.282646 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:21.282648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:21.283399 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:21.283401 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:21.284219 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:21.284222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:21.284590 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:21.284591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:21.284643 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:21.285212 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:21.285219 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:21.287339 (+ 2120us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":90,"threads_started":1}
W20260619 20:32:21.287662 13578 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:21.281999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:33697 (local address 127.4.83.126:42463)
0619 20:32:21.282073 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:21.282075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:21.282195 (+ 120us) server_negotiation.cc:407] Connection header received
0619 20:32:21.282256 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:21.282257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:21.282272 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:21.282327 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:21.282714 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:21.283272 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:21.284296 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:21.284514 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:21.284572 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:21.287577 (+ 3005us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.66:33697: BlockingRecv error: recv got EOF from 127.4.83.66:33697 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Jun 19 20:32:21 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:21 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:21.422160 13580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:21.416801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.67:49603)
0619 20:32:21.416961 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:21.416974 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:21.417064 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:21.417244 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:21.417245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:21.417456 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:21.417520 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:21.417522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:21.418275 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:21.418277 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:21.419043 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:21.419047 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:21.419456 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:21.419458 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:21.419512 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:21.420081 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:21.420086 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:21.422062 (+ 1976us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":91,"threads_started":1}
W20260619 20:32:21.422326 13578 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:21.416945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:49603 (local address 127.4.83.126:42463)
0619 20:32:21.417022 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:21.417024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:21.417042 (+ 18us) server_negotiation.cc:407] Connection header received
0619 20:32:21.417140 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:21.417141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:21.417158 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:21.417233 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:21.417602 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:21.418166 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:21.419146 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:21.419361 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:21.419444 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:21.422250 (+ 2806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.67:49603: BlockingRecv error: recv got EOF from 127.4.83.67:49603 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Jun 19 20:32:22 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:22 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:22.145700 13581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:22.139764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.65:50133)
0619 20:32:22.139986 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:22.140002 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:22.140090 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:22.140237 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:22.140237 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:22.140459 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:22.140525 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:22.140528 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:22.141295 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:22.141297 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:22.142008 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:22.142013 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:22.142454 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:22.142454 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:22.142502 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:22.143182 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:22.143188 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:22.145594 (+ 2406us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":119,"threads_started":1}
W20260619 20:32:22.145897 13582 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:22.139958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:50133 (local address 127.4.83.126:42463)
0619 20:32:22.140108 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:22.140110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:22.140118 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:32:22.140147 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:22.140147 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:22.140159 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:22.140225 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:22.140594 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:22.141185 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:22.142140 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:22.142377 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:22.142454 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:22.145821 (+ 3367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.65:50133: BlockingRecv error: recv got EOF from 127.4.83.65:50133 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":79,"threads_started":1}
Jun 19 20:32:22 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:22 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:22.293977 13583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:22.288114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.66:53357)
0619 20:32:22.288289 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:22.288302 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:22.288371 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:22.288524 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:22.288525 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:22.288716 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:22.288783 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:22.288785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:22.289582 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:22.289584 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:22.290212 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:22.290215 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:22.290935 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:22.290937 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:22.290997 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:22.291762 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:22.291768 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:22.293876 (+ 2108us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":104,"threads_started":1}
W20260619 20:32:22.294104 13582 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:22.288250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:53357 (local address 127.4.83.126:42463)
0619 20:32:22.288354 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:22.288356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:22.288369 (+ 13us) server_negotiation.cc:407] Connection header received
0619 20:32:22.288430 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:22.288431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:22.288445 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:22.288510 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:22.288866 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:22.289447 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:22.290515 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:22.290767 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:22.290860 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:22.294042 (+ 3182us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.66:53357: BlockingRecv error: recv got EOF from 127.4.83.66:53357 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Jun 19 20:32:22 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:22 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:22.427455 13584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:22.422718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.67:57979)
0619 20:32:22.422855 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:22.422867 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:22.422937 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:22.423050 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:22.423051 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:22.423242 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:22.423313 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:22.423316 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:22.424037 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:22.424038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:22.424608 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:22.424610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:22.424944 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:22.424944 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:22.424984 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:22.425492 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:22.425496 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:22.427381 (+ 1885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":75,"threads_started":1}
W20260619 20:32:22.427568 13582 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:22.422764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:57979 (local address 127.4.83.126:42463)
0619 20:32:22.422835 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:22.422837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:22.422918 (+ 81us) server_negotiation.cc:407] Connection header received
0619 20:32:22.422982 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:22.422983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:22.422997 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:22.423043 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:22.423365 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:22.423974 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:22.424687 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:22.424885 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:22.424942 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:22.427491 (+ 2549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.67:57979: BlockingRecv error: recv got EOF from 127.4.83.67:57979 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Jun 19 20:32:23 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:23 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:23.151664 13585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:23.146461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.65:36861)
0619 20:32:23.146653 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:23.146671 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:23.146761 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:23.146954 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:23.146955 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:23.147152 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:23.147221 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:23.147223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:23.148027 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:23.148029 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:23.148720 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:23.148723 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:23.149086 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:23.149087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:23.149145 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:23.149698 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:23.149703 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:23.151582 (+ 1879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":94,"threads_started":1}
W20260619 20:32:23.151816 13586 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:23.146602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:36861 (local address 127.4.83.126:42463)
0619 20:32:23.146773 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:23.146775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:23.146793 (+ 18us) server_negotiation.cc:407] Connection header received
0619 20:32:23.146826 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:23.146826 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:23.146839 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:23.146907 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:23.147290 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:23.147951 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:23.148807 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:23.149023 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:23.149082 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:23.151734 (+ 2652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.65:36861: BlockingRecv error: recv got EOF from 127.4.83.65:36861 (error 108)
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":98,"threads_started":1}
Jun 19 20:32:23 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:23 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:23.304380 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:23.294787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.66:44473)
0619 20:32:23.294977 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:23.294992 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:23.295069 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:23.295263 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:23.295264 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:23.295507 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:23.295588 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:23.295591 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:23.296463 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:23.296464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:23.297088 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:23.297090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:23.298121 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:23.298122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:23.298200 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:23.299784 (+ 1584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:23.299802 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:23.304079 (+ 4277us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":82,"threads_started":1}
W20260619 20:32:23.304584 13586 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:23.295023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:44473 (local address 127.4.83.126:42463)
0619 20:32:23.295094 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:23.295096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:23.295106 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:32:23.295161 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:23.295161 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:23.295176 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:23.295230 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:23.295688 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:23.296361 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:23.297179 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:23.297973 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:23.298087 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:23.304523 (+ 6436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.66:44473: BlockingRecv error: recv got EOF from 127.4.83.66:44473 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Jun 19 20:32:23 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:23 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:23.433868 13588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:23.428238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.67:46265)
0619 20:32:23.428420 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:23.428432 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:23.428507 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:23.428844 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:23.428845 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:23.429051 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:23.429118 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:23.429120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:23.429911 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:23.429912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:23.430540 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:23.430544 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:23.431008 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:23.431010 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:23.431065 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:23.431602 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:23.431606 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:23.433793 (+ 2187us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":97,"threads_started":1}
W20260619 20:32:23.434052 13586 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:23.428391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:46265 (local address 127.4.83.126:42463)
0619 20:32:23.428451 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:23.428453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:23.428595 (+ 142us) server_negotiation.cc:407] Connection header received
0619 20:32:23.428641 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:23.428643 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:23.428662 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:23.428817 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:23.429193 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:23.429747 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:23.430667 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:23.430925 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:23.430991 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:23.433980 (+ 2989us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.67:46265: BlockingRecv error: recv got EOF from 127.4.83.67:46265 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Jun 19 20:32:24 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:24 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:24.160894 13589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:24.152361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.65:44031)
0619 20:32:24.152512 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:24.152530 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:24.152622 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:24.152774 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:24.152775 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:24.152969 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:24.153034 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:24.153036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:24.153779 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:24.153780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:24.154428 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:24.154432 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:24.154882 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:24.154883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:24.154955 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:24.158666 (+ 3711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:24.158673 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:24.160814 (+ 2141us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
W20260619 20:32:24.161067 13590 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:24.152478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:44031 (local address 127.4.83.126:42463)
0619 20:32:24.152631 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:24.152633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:24.152645 (+ 12us) server_negotiation.cc:407] Connection header received
0619 20:32:24.152680 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:24.152681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:24.152694 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:24.152758 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:24.153111 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:24.153694 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:24.154517 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:24.154772 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:24.154844 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:24.160974 (+ 6130us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.65:44031: BlockingRecv error: recv got EOF from 127.4.83.65:44031 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":82,"threads_started":1}
Jun 19 20:32:24 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:24 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:24.310581 13591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:24.305138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.66:41569)
0619 20:32:24.305317 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:24.305330 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:24.305415 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:24.305594 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:24.305595 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:24.305797 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:24.305934 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:24.305937 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:24.306665 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:24.306687 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:24.307248 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:24.307250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:24.307695 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:24.307696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:24.307761 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:24.308360 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:24.308366 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:24.310485 (+ 2119us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":94,"threads_started":1}
W20260619 20:32:24.310766 13590 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:24.305298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:41569 (local address 127.4.83.126:42463)
0619 20:32:24.305364 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:24.305366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:24.305394 (+ 28us) server_negotiation.cc:407] Connection header received
0619 20:32:24.305478 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:24.305479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:24.305493 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:24.305544 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:24.306024 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:24.306581 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:24.307344 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:24.307598 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:24.307668 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:24.310631 (+ 2963us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.66:41569: BlockingRecv error: recv got EOF from 127.4.83.66:41569 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Jun 19 20:32:24 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:24 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:24.440165 13592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:24.434502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.67:43999)
0619 20:32:24.434676 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:24.434692 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:24.434803 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:24.434961 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:24.434962 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:24.435217 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:24.435301 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:24.435304 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:24.436105 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:24.436107 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:24.436706 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:24.436709 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:24.437141 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:24.437142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:24.437193 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:24.437825 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:24.437830 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:24.440065 (+ 2235us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":85,"threads_started":1}
W20260619 20:32:24.440267 13590 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:24.434643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:43999 (local address 127.4.83.126:42463)
0619 20:32:24.434710 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:24.434712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:24.434794 (+ 82us) server_negotiation.cc:407] Connection header received
0619 20:32:24.434866 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:24.434867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:24.434887 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:24.434945 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:24.435393 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:24.435988 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:24.436854 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:24.437070 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:24.437136 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:24.440202 (+ 3066us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.67:43999: BlockingRecv error: recv got EOF from 127.4.83.67:43999 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Jun 19 20:32:25 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:25 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:25.168578 13593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:25.161640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.65:39645)
0619 20:32:25.161797 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:25.161810 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:25.161914 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:25.162218 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:25.162220 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:25.162463 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:25.162536 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:25.162538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:25.163370 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:25.163372 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:25.164034 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:25.164036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:25.164679 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:25.164680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:25.164825 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:25.165606 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:25.165614 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:25.168489 (+ 2875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
W20260619 20:32:25.168844 13594 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:25.161664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:39645 (local address 127.4.83.126:42463)
0619 20:32:25.161957 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:25.161961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:25.161973 (+ 12us) server_negotiation.cc:407] Connection header received
0619 20:32:25.162013 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:25.162013 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:25.162029 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:25.162120 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:25.162667 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:25.163273 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:25.164185 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:25.164592 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:25.164681 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:25.168752 (+ 4071us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.65:39645: BlockingRecv error: recv got EOF from 127.4.83.65:39645 (error 108)
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":191,"threads_started":1}
Jun 19 20:32:25 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:25 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:25.316884 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:25.311337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.66:50511)
0619 20:32:25.311590 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:25.311603 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:25.311722 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:25.311986 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:25.311988 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:25.312235 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:25.312304 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:25.312307 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:25.313087 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:25.313088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:25.313673 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:25.313675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:25.314093 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:25.314094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:25.314154 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:25.314792 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:25.314797 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:25.316786 (+ 1989us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":91,"threads_started":1}
W20260619 20:32:25.316995 13594 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:25.311507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:50511 (local address 127.4.83.126:42463)
0619 20:32:25.311585 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:25.311588 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:25.311716 (+ 128us) server_negotiation.cc:407] Connection header received
0619 20:32:25.311796 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:25.311797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:25.311812 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:25.311906 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:25.312411 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:25.312971 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:25.313800 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:25.314018 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:25.314086 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:25.316934 (+ 2848us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.66:50511: BlockingRecv error: recv got EOF from 127.4.83.66:50511 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Jun 19 20:32:25 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:25 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:25.446080 13596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:25.440847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.67:52179)
0619 20:32:25.441022 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:25.441039 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:25.441136 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:25.441306 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:25.441307 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:25.441552 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:25.441617 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:25.441619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:25.442397 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:25.442398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:25.442972 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:25.442974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:25.443320 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:25.443321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:25.443376 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:25.443972 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:25.443977 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:25.445984 (+ 2007us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":88,"threads_started":1}
W20260619 20:32:25.446201 13594 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:25.440995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:52179 (local address 127.4.83.126:42463)
0619 20:32:25.441107 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:25.441109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:25.441130 (+ 21us) server_negotiation.cc:407] Connection header received
0619 20:32:25.441219 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:25.441220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:25.441237 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:25.441308 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:25.441707 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:25.442312 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:25.443059 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:25.443251 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:25.443320 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:25.446134 (+ 2814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.67:52179: BlockingRecv error: recv got EOF from 127.4.83.67:52179 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Jun 19 20:32:26 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:26 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:26.175420 13597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:26.169196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.65:57537)
0619 20:32:26.169383 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:26.169395 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:26.169500 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:26.169712 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:26.169713 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:26.169986 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:26.170055 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:26.170057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:26.170903 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:26.170905 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:26.171612 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:26.171616 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:26.172191 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:26.172192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:26.172259 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:26.172825 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:26.172830 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:26.175335 (+ 2505us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":84,"threads_started":1}
W20260619 20:32:26.175654 13598 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:26.169378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:57537 (local address 127.4.83.126:42463)
0619 20:32:26.169518 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:26.169521 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:26.169559 (+ 38us) server_negotiation.cc:407] Connection header received
0619 20:32:26.169592 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:26.169592 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:26.169605 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:26.169668 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:26.170206 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:26.170778 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:26.171782 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:26.172049 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:26.172162 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:26.175562 (+ 3400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.65:57537: BlockingRecv error: recv got EOF from 127.4.83.65:57537 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":65,"threads_started":1}
Jun 19 20:32:26 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:26 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:26.323980 13599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:26.317622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.66:38327)
0619 20:32:26.317801 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:26.317813 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:26.317897 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:26.318103 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:26.318105 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:26.318343 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:26.318413 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:26.318415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:26.319441 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:26.319443 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:26.320199 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:26.320204 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:26.320693 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:26.320694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:26.320765 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:26.321313 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:26.321320 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:26.323883 (+ 2563us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":92,"threads_started":1}
W20260619 20:32:26.324167 13598 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:26.317692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:38327 (local address 127.4.83.126:42463)
0619 20:32:26.317757 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:26.317759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:26.317886 (+ 127us) server_negotiation.cc:407] Connection header received
0619 20:32:26.317966 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:26.317967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:26.317980 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:26.318057 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:26.318529 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:26.319321 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:26.320305 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:26.320601 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:26.320677 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:26.324085 (+ 3408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.66:38327: BlockingRecv error: recv got EOF from 127.4.83.66:38327 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Jun 19 20:32:26 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:26 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:26.452267 13600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:26.446822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.67:41167)
0619 20:32:26.447022 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:26.447038 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:26.447141 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:26.447295 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:26.447295 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:26.447515 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:26.447580 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:26.447583 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:26.448374 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:26.448375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:26.448935 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:26.448937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:26.449280 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:26.449280 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:26.449335 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:26.449965 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:26.449970 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:26.452133 (+ 2163us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":96,"threads_started":1}
W20260619 20:32:26.452351 13598 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:26.446943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:41167 (local address 127.4.83.126:42463)
0619 20:32:26.447038 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:26.447041 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:26.447128 (+ 87us) server_negotiation.cc:407] Connection header received
0619 20:32:26.447207 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:26.447209 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:26.447226 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:26.447279 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:26.447679 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:26.448286 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:26.449015 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:26.449216 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:26.449275 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:26.452288 (+ 3013us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.67:41167: BlockingRecv error: recv got EOF from 127.4.83.67:41167 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Jun 19 20:32:27 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:27 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:27.182186 13601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:27.176095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.65:58901)
0619 20:32:27.176277 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:27.176290 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:27.176356 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:27.176600 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:27.176601 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:27.176827 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:27.176898 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:27.176901 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:27.177677 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:27.177678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:27.178265 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:27.178266 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:27.178675 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:27.178675 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:27.178726 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:27.179284 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:27.179288 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:27.182085 (+ 2797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":107,"threads_started":1}
W20260619 20:32:27.182323 13602 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:27.176230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:58901 (local address 127.4.83.126:42463)
0619 20:32:27.176409 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:27.176411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:27.176424 (+ 13us) server_negotiation.cc:407] Connection header received
0619 20:32:27.176457 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:27.176457 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:27.176476 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:27.176568 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:27.177015 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:27.177564 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:27.178370 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:27.178579 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:27.178646 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:27.182235 (+ 3589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.65:58901: BlockingRecv error: recv got EOF from 127.4.83.65:58901 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":80,"threads_started":1}
Jun 19 20:32:27 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:27 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:27.330027 13603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:27.324604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.66:58055)
0619 20:32:27.324752 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:27.324769 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:27.324834 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:27.324973 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:27.324974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:27.325153 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:27.325215 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:27.325217 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:27.325884 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:27.325885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:27.326466 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:27.326468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:27.326790 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:27.326790 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:27.326842 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:27.327319 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:27.327324 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:27.329938 (+ 2614us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":64,"threads_started":1}
W20260619 20:32:27.330106 13602 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:27.324756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:58055 (local address 127.4.83.126:42463)
0619 20:32:27.324847 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:27.324849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:27.324860 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:32:27.324895 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:27.324896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:27.324910 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:27.324959 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:27.325284 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:27.325809 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:27.326543 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:27.326730 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:27.326788 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:27.330054 (+ 3266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.66:58055: BlockingRecv error: recv got EOF from 127.4.83.66:58055 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 19 20:32:27 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:27 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:27.458182 13604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:27.452816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.67:58925)
0619 20:32:27.453020 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:27.453035 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:27.453158 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:27.453304 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:27.453304 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:27.453504 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:27.453575 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:27.453577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:27.454214 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:27.454215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:27.454838 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:27.454840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:27.455271 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:27.455272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:27.455338 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:27.455960 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:27.455966 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:27.458098 (+ 2132us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":70,"threads_started":1}
W20260619 20:32:27.458307 13602 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:27.452983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:58925 (local address 127.4.83.126:42463)
0619 20:32:27.453042 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:27.453044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:27.453146 (+ 102us) server_negotiation.cc:407] Connection header received
0619 20:32:27.453225 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:27.453225 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:27.453238 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:27.453278 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:27.453633 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:27.454129 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:27.454956 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:27.455146 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:27.455216 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:27.458230 (+ 3014us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.67:58925: BlockingRecv error: recv got EOF from 127.4.83.67:58925 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Jun 19 20:32:28 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:28 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:28.188720 13605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:28.182719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.65:52045)
0619 20:32:28.182895 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:28.182908 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:28.182983 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:28.183132 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:28.183133 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:28.183333 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:28.183397 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:28.183398 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:28.184465 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:28.184467 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:28.185060 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:28.185062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:28.185538 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:28.185540 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:28.185601 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:28.186239 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:28.186245 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:28.188616 (+ 2371us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":91,"threads_started":1}
W20260619 20:32:28.188830 13606 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:28.182793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:52045 (local address 127.4.83.126:42463)
0619 20:32:28.182914 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:28.182916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:28.182974 (+ 58us) server_negotiation.cc:407] Connection header received
0619 20:32:28.183034 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:28.183035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:28.183046 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:28.183121 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:28.183516 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:28.184349 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:28.185171 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:28.185394 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:28.185466 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:28.188757 (+ 3291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.65:52045: BlockingRecv error: recv got EOF from 127.4.83.65:52045 (error 108)
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":56,"threads_started":1}
Jun 19 20:32:28 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:28 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:28.336280 13607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:28.330677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.66:55199)
0619 20:32:28.330898 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:28.330912 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:28.330997 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:28.331156 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:28.331157 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:28.331378 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:28.331442 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:28.331444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:28.332229 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:28.332230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:28.332817 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:28.332819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:28.333375 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:28.333377 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:28.333434 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:28.334057 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:28.334068 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:28.336200 (+ 2132us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":120,"threads_started":1}
W20260619 20:32:28.336504 13606 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:28.330775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:55199 (local address 127.4.83.126:42463)
0619 20:32:28.330879 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:28.330882 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:28.330984 (+ 102us) server_negotiation.cc:407] Connection header received
0619 20:32:28.331070 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:28.331072 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:28.331086 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:28.331140 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:28.331523 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:28.332134 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:28.332944 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:28.333258 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:28.333343 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:28.336433 (+ 3090us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.66:55199: BlockingRecv error: recv got EOF from 127.4.83.66:55199 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 19 20:32:28 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:28 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:28.465025 13608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:28.458866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.67:41713)
0619 20:32:28.459094 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:28.459109 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:28.459192 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:28.459363 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:28.459364 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:28.459541 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:28.459596 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:28.459598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:28.460636 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:28.460638 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:28.461175 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:28.461177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:28.461644 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:28.461647 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:28.461738 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:28.462347 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:28.462353 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:28.464937 (+ 2584us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":91,"threads_started":1}
W20260619 20:32:28.465109 13606 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:28.459152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:41713 (local address 127.4.83.126:42463)
0619 20:32:28.459228 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:28.459231 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:28.459239 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:32:28.459269 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:28.459270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:28.459283 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:28.459339 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:28.459673 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:28.460550 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:28.461252 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:28.461453 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:28.461518 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:28.465054 (+ 3536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.67:41713: BlockingRecv error: recv got EOF from 127.4.83.67:41713 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Jun 19 20:32:29 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:29 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:29.199586 13609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:29.189436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.65:48705)
0619 20:32:29.189595 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:29.189607 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:29.189696 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:29.190048 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:29.190049 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:29.190222 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:29.190282 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:29.190284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:29.191108 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:29.191109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:29.191714 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:29.191716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:29.192233 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:29.192234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:29.192310 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:29.197260 (+ 4950us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:29.197267 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:29.199486 (+ 2219us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":76,"threads_started":1}
W20260619 20:32:29.199812 13610 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:29.189560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:48705 (local address 127.4.83.126:42463)
0619 20:32:29.189816 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:29.189819 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:29.189836 (+ 17us) server_negotiation.cc:407] Connection header received
0619 20:32:29.189882 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:29.189883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:29.189899 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:29.190023 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:29.190384 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:29.190971 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:29.191868 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:29.192123 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:29.192204 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:29.199704 (+ 7500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.65:48705: BlockingRecv error: recv got EOF from 127.4.83.65:48705 (error 108)
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":119,"threads_started":1}
Jun 19 20:32:29 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:29 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:29.344349 13611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:29.336951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.66:50943)
0619 20:32:29.337189 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:29.337207 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:29.337334 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:29.337747 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:29.337748 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:29.338005 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:29.338073 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:29.338075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:29.339202 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:29.339204 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:29.340216 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:29.340219 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:29.340857 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:29.340859 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:29.340940 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:29.341672 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:29.341679 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:29.344242 (+ 2563us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":122,"threads_started":1}
W20260619 20:32:29.344518 13610 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:29.337128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:50943 (local address 127.4.83.126:42463)
0619 20:32:29.337224 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:29.337227 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:29.337373 (+ 146us) server_negotiation.cc:407] Connection header received
0619 20:32:29.337463 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:29.337464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:29.337483 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:29.337556 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:29.338224 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:29.339080 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:29.340338 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:29.340675 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:29.340764 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:29.344433 (+ 3669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.66:50943: BlockingRecv error: recv got EOF from 127.4.83.66:50943 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Jun 19 20:32:29 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:29 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:29.471331 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:29.465808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.67:35495)
0619 20:32:29.465995 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:29.466008 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:29.466086 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:29.466268 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:29.466269 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:29.466468 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:29.466534 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:29.466536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:29.467239 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:29.467240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:29.467903 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:29.467905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:29.468298 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:29.468299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:29.468370 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:29.468938 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:29.468944 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:29.471221 (+ 2277us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":106,"threads_started":1}
W20260619 20:32:29.471578 13610 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:29.465915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:35495 (local address 127.4.83.126:42463)
0619 20:32:29.465973 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:29.465975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:29.466082 (+ 107us) server_negotiation.cc:407] Connection header received
0619 20:32:29.466153 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:29.466153 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:29.466167 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:29.466248 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:29.466639 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:29.467147 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:29.467975 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:29.468163 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:29.468225 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:29.471386 (+ 3161us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.67:35495: BlockingRecv error: recv got EOF from 127.4.83.67:35495 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Jun 19 20:32:30 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:30 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:30.206663 13613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:30.200219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.65:49907)
0619 20:32:30.200381 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:30.200391 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:30.200447 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:30.200706 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:30.200708 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:30.200940 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:30.201012 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:30.201014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:30.201949 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:30.201950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:30.202579 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:30.202582 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:30.203089 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:30.203090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:30.203152 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:30.203895 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:30.203903 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:30.206562 (+ 2659us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":67,"threads_started":1}
W20260619 20:32:30.206836 13614 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:30.200370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:49907 (local address 127.4.83.126:42463)
0619 20:32:30.200493 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:30.200495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:30.200504 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:32:30.200534 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:30.200534 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:30.200546 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:30.200620 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:30.201167 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:30.201777 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:30.202705 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:30.202974 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:30.203057 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:30.206744 (+ 3687us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.65:49907: BlockingRecv error: recv got EOF from 127.4.83.65:49907 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":57,"threads_started":1}
Jun 19 20:32:30 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:30 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:30.350587 13615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:30.345237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.66:44831)
0619 20:32:30.345393 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:30.345406 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:30.345483 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:30.345616 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:30.345616 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:30.345819 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:30.345878 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:30.345881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:30.346580 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:30.346581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:30.347271 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:30.347275 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:30.347697 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:30.347698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:30.347747 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:30.348357 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:30.348362 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:30.350507 (+ 2145us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":82,"threads_started":1}
W20260619 20:32:30.350785 13614 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:30.345348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:44831 (local address 127.4.83.126:42463)
0619 20:32:30.345413 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:30.345415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:30.345469 (+ 54us) server_negotiation.cc:407] Connection header received
0619 20:32:30.345542 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:30.345543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:30.345555 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:30.345601 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:30.345932 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:30.346489 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:30.347390 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:30.347625 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:30.347697 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:30.350710 (+ 3013us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.66:44831: BlockingRecv error: recv got EOF from 127.4.83.66:44831 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 19 20:32:30 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:30 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:30.477257 13616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:30.472074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.67:33273)
0619 20:32:30.472255 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:30.472267 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:30.472332 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:30.472491 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:30.472492 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:30.472695 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:30.472757 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:30.472759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:30.473497 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:30.473498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:30.474085 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:30.474087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:30.474491 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:30.474491 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:30.474537 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:30.475110 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:30.475116 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:30.477158 (+ 2042us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":109,"threads_started":1}
W20260619 20:32:30.477408 13614 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:30.472206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:33273 (local address 127.4.83.126:42463)
0619 20:32:30.472306 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:30.472308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:30.472322 (+ 14us) server_negotiation.cc:407] Connection header received
0619 20:32:30.472394 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:30.472395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:30.472410 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:30.472471 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:30.472818 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:30.473395 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:30.474199 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:30.474416 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:30.474480 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:30.477344 (+ 2864us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.67:33273: BlockingRecv error: recv got EOF from 127.4.83.67:33273 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Jun 19 20:32:31 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:31 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:31.212013 13617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:31.207242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.65:56819)
0619 20:32:31.207425 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:31.207436 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:31.207485 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:31.207632 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:31.207633 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:31.207816 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:31.207908 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:31.207911 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:31.208604 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:31.208604 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:31.209146 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:31.209147 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:31.209459 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:31.209459 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:31.209497 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:31.209993 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:31.209997 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:31.211948 (+ 1951us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":64,"threads_started":1}
W20260619 20:32:31.212088 13618 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:31.207381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:56819 (local address 127.4.83.126:42463)
0619 20:32:31.207497 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:31.207499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:31.207509 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:32:31.207541 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:31.207542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:31.207553 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:31.207620 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:31.207995 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:31.208540 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:31.209217 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:31.209404 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:31.209457 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:31.212025 (+ 2568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.65:56819: BlockingRecv error: recv got EOF from 127.4.83.65:56819 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":55,"threads_started":1}
Jun 19 20:32:31 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:31 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:31.355636 13619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:31.351150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.66:34331)
0619 20:32:31.351266 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:31.351277 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:31.351335 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:31.351463 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:31.351464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:31.351628 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:31.351678 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:31.351679 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:31.352326 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:31.352327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:31.352906 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:31.352908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:31.353235 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:31.353236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:31.353271 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:31.353706 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:31.353711 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:31.355549 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":60,"threads_started":1}
W20260619 20:32:31.355772 13618 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:31.351271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:34331 (local address 127.4.83.126:42463)
0619 20:32:31.351318 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:31.351320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:31.351332 (+ 12us) server_negotiation.cc:407] Connection header received
0619 20:32:31.351386 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:31.351386 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:31.351397 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:31.351451 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:31.351731 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:31.352266 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:31.352988 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:31.353180 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:31.353235 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:31.355709 (+ 2474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.66:34331: BlockingRecv error: recv got EOF from 127.4.83.66:34331 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Jun 19 20:32:31 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:31 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:31.482465 13620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:31.477793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.67:37399)
0619 20:32:31.477966 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:31.477980 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:31.478045 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:31.478202 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:31.478203 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:31.478369 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:31.478439 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:31.478442 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:31.479146 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:31.479146 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:31.479744 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:31.479745 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:31.480133 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:31.480134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:31.480178 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:31.480634 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:31.480638 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:31.482386 (+ 1748us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":76,"threads_started":1}
W20260619 20:32:31.482555 13618 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:31.477952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:37399 (local address 127.4.83.126:42463)
0619 20:32:31.478030 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:31.478035 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:31.478046 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:32:31.478092 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:31.478092 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:31.478105 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:31.478162 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:31.478551 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:31.479061 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:31.479828 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:31.480066 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:31.480126 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:31.482490 (+ 2364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.67:37399: BlockingRecv error: recv got EOF from 127.4.83.67:37399 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 19 20:32:32 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:32 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:32.218145 13621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:32.212557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.65:42199)
0619 20:32:32.212817 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:32.212834 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:32.212925 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:32.213088 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:32.213089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:32.213308 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:32.213402 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:32.213404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:32.214081 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:32.214082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:32.214713 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:32.214715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:32.215106 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:32.215107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:32.215165 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:32.215760 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:32.215764 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:32.218066 (+ 2302us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":75,"threads_started":1}
W20260619 20:32:32.218304 13622 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:32.212659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:42199 (local address 127.4.83.126:42463)
0619 20:32:32.212779 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:32.212781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:32.212905 (+ 124us) server_negotiation.cc:407] Connection header received
0619 20:32:32.212984 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:32.212984 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:32.212996 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:32.213063 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:32.213476 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:32.213987 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:32.214813 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:32.215022 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:32.215081 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:32.218196 (+ 3115us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.65:42199: BlockingRecv error: recv got EOF from 127.4.83.65:42199 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":58,"threads_started":1}
Jun 19 20:32:32 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:32 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:32.362255 13623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:32.356418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.66:47653)
0619 20:32:32.356619 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:32.356634 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:32.356748 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:32.356908 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:32.356909 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:32.357265 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:32.357331 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:32.357334 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:32.358148 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:32.358149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:32.358810 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:32.358813 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:32.359213 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:32.359214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:32.359269 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:32.359960 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:32.359969 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:32.362177 (+ 2208us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":79,"threads_started":1}
W20260619 20:32:32.362475 13622 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:32.356510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:47653 (local address 127.4.83.126:42463)
0619 20:32:32.356601 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:32.356605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:32.356733 (+ 128us) server_negotiation.cc:407] Connection header received
0619 20:32:32.356828 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:32.356830 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:32.356843 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:32.356894 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:32.357475 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:32.358038 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:32.358923 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:32.359139 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:32.359210 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:32.362385 (+ 3175us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.66:47653: BlockingRecv error: recv got EOF from 127.4.83.66:47653 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Jun 19 20:32:32 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:32 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.67@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:32.488684 13624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:32.483038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.67:55321)
0619 20:32:32.483175 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:32.483187 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:32.483250 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:32.483573 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:32.483574 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:32.483786 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:32.483884 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:32.483887 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:32.484868 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:32.484869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:32.485454 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:32.485456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:32.485887 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:32.485888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:32.485935 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:32.486487 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:32.486491 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:32.488578 (+ 2087us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":76,"threads_started":1}
W20260619 20:32:32.488814 13622 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:32.483180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:55321 (local address 127.4.83.126:42463)
0619 20:32:32.483228 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:32.483230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:32.483239 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:32:32.483300 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:32.483300 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:32.483425 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:32.483578 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:32.484023 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:32.484769 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:32.485561 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:32.485804 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:32.485873 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:32.488737 (+ 2864us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.67:55321: BlockingRecv error: recv got EOF from 127.4.83.67:55321 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Jun 19 20:32:33 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:33 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.65@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:33.225041 13625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:33.218942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.65:45829)
0619 20:32:33.219090 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:33.219103 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:33.219160 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:33.219564 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:33.219566 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:33.219950 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:33.220013 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:33.220016 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:33.220976 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:33.220976 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:33.221542 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:33.221544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:33.221952 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:33.221953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:33.221995 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:33.222570 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:33.222575 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:33.224923 (+ 2348us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
W20260619 20:32:33.225152 13626 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:33.219089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:45829 (local address 127.4.83.126:42463)
0619 20:32:33.219279 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:33.219281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:33.219296 (+ 15us) server_negotiation.cc:407] Connection header received
0619 20:32:33.219333 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:33.219334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:33.219356 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:33.219467 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:33.220141 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:33.220843 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:33.221639 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:33.221870 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:33.221955 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:33.225079 (+ 3124us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.65:45829: BlockingRecv error: recv got EOF from 127.4.83.65:45829 (error 108)
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":87,"threads_started":1}
Jun 19 20:32:33 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:32:33 dist-test-slave-4m58 krb5kdc[12567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.83.66@KRBTEST.COM for oryx/127.4.83.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:32:33.367551 13627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:33.362891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42463 (local address 127.4.83.66:34425)
0619 20:32:33.363062 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:33.363076 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:33.363164 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:33.363365 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:33.363367 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:33.363527 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:33.363581 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:33.363583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:33.364361 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:33.364362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:33.364933 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:33.364934 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:33.365311 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:33.365312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:33.365366 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:33.365850 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:33.365855 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:33.367490 (+ 1635us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.83.126:42463: Server oryx/127.4.83.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
W20260619 20:32:33.367729 13626 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:33.363015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:34425 (local address 127.4.83.126:42463)
0619 20:32:33.363107 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:33.363110 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:33.363140 (+ 30us) server_negotiation.cc:407] Connection header received
0619 20:32:33.363275 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:33.363277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:33.363294 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:33.363355 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:33.363648 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:33.364260 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:33.365009 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:33.365220 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:33.365271 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:33.367641 (+ 2370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.83.66:34425: BlockingRecv error: recv got EOF from 127.4.83.66:34425 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 20:32:33.402697 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 13150
I20260619 20:32:33.407979 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 13302
I20260619 20:32:33.412647 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 13436
I20260619 20:32:33.417096 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 13080
2026-06-19T20:32:33Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17504 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-19T20:32:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:32:33Z Disabled control of system clock
I20260619 20:32:33.446230 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:35475
--webserver_interface=127.4.83.126
--webserver_port=0
--builtin_ntp_servers=127.4.83.84:36309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:35475
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 20:32:33.529348 13631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:33.529589 13631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:33.529609 13631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:33.530855 13631 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:32:33.530908 13631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:33.530920 13631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:33.530931 13631 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:32:33.530942 13631 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:32:33.532445 13631 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:36309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:35475
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:35475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:33.532691 13631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:33.532945 13631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:33.539299 13636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:33.539410 13637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:33.539465 13631 server_base.cc:1034] running on GCE node
W20260619 20:32:33.539535 13639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:33.540028 13631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:33.540374 13631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:33.541563 13631 hybrid_clock.cc:648] HybridClock initialized: now 1781901153541543 us; error 36 us; skew 500 ppm
I20260619 20:32:33.543138 13631 webserver.cc:460] Webserver started at http://127.4.83.126:41883/ using document root <none> and password file <none>
I20260619 20:32:33.543427 13631 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:33.543489 13631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:33.543589 13631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:33.544737 13631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "68749e04622047f3afbe571842e56878"
format_stamp: "Formatted at 2026-06-19 20:32:33 on dist-test-slave-4m58"
server_key: "3c252adaec83113616ea8434aad8d6d6"
server_key_iv: "f1d3ff7d47bc4e136402f52aca8b5267"
server_key_version: "encryptionkey@0"
I20260619 20:32:33.545076 13631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "68749e04622047f3afbe571842e56878"
format_stamp: "Formatted at 2026-06-19 20:32:33 on dist-test-slave-4m58"
server_key: "3c252adaec83113616ea8434aad8d6d6"
server_key_iv: "f1d3ff7d47bc4e136402f52aca8b5267"
server_key_version: "encryptionkey@0"
I20260619 20:32:33.546547 13631 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 20:32:33.547370 13645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:33.547612 13631 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:33.547708 13631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "68749e04622047f3afbe571842e56878"
format_stamp: "Formatted at 2026-06-19 20:32:33 on dist-test-slave-4m58"
server_key: "3c252adaec83113616ea8434aad8d6d6"
server_key_iv: "f1d3ff7d47bc4e136402f52aca8b5267"
server_key_version: "encryptionkey@0"
I20260619 20:32:33.547775 13631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:33.582185 13631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:33.582548 13631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:33.582688 13631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:33.587965 13631 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:35475
I20260619 20:32:33.588074 13697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:35475 every 8 connection(s)
I20260619 20:32:33.588511 13631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:32:33.589627 13698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:33.591921 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 13631
I20260619 20:32:33.592051 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:32:33.592280 4429 external_mini_cluster.cc:1428] Setting key 160f00f0c6a93b1c3cc0ae1e80f2fcfc
I20260619 20:32:33.593520 13698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68749e04622047f3afbe571842e56878: Bootstrap starting.
I20260619 20:32:33.594358 13698 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68749e04622047f3afbe571842e56878: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:33.594697 13698 log.cc:824] T 00000000000000000000000000000000 P 68749e04622047f3afbe571842e56878: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:33.595727 13698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68749e04622047f3afbe571842e56878: No bootstrap required, opened a new log
I20260619 20:32:33.598109 13698 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68749e04622047f3afbe571842e56878 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68749e04622047f3afbe571842e56878" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35475 } }
I20260619 20:32:33.598294 13698 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68749e04622047f3afbe571842e56878 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:33.598330 13698 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68749e04622047f3afbe571842e56878 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68749e04622047f3afbe571842e56878, State: Initialized, Role: FOLLOWER
I20260619 20:32:33.598520 13698 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68749e04622047f3afbe571842e56878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68749e04622047f3afbe571842e56878" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35475 } }
I20260619 20:32:33.598639 13698 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68749e04622047f3afbe571842e56878 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:32:33.598682 13698 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68749e04622047f3afbe571842e56878 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:32:33.598760 13698 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68749e04622047f3afbe571842e56878 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:33.599049 13701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:33.592992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59960 (local address 127.4.83.126:35475)
0619 20:32:33.593210 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:33.593221 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:33.593257 (+ 36us) server_negotiation.cc:407] Connection header received
0619 20:32:33.593978 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:33.593985 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:33.594064 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:33.594201 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:33.594447 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:33.595645 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:33.596546 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:33.597042 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:33.597175 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:33.598103 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:33.598107 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:33.598107 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:32:33.598287 (+ 180us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:33.598330 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:32:33.598345 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:33.598447 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:33.598610 (+ 163us) server_negotiation.cc:299] Negotiation successful
0619 20:32:33.598842 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":75,"threads_started":1}
I20260619 20:32:33.599591 13698 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68749e04622047f3afbe571842e56878 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68749e04622047f3afbe571842e56878" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35475 } }
I20260619 20:32:33.599720 13698 leader_election.cc:302] T 00000000000000000000000000000000 P 68749e04622047f3afbe571842e56878 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68749e04622047f3afbe571842e56878; no voters:
I20260619 20:32:33.599941 13698 leader_election.cc:288] T 00000000000000000000000000000000 P 68749e04622047f3afbe571842e56878 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:32:33.599994 13703 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68749e04622047f3afbe571842e56878 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:32:33.600142 13703 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68749e04622047f3afbe571842e56878 [term 1 LEADER]: Becoming Leader. State: Replica: 68749e04622047f3afbe571842e56878, State: Running, Role: LEADER
I20260619 20:32:33.600284 13703 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68749e04622047f3afbe571842e56878 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68749e04622047f3afbe571842e56878" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35475 } }
I20260619 20:32:33.600334 13698 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68749e04622047f3afbe571842e56878 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:32:33.600785 13705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68749e04622047f3afbe571842e56878 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68749e04622047f3afbe571842e56878. Latest consensus state: current_term: 1 leader_uuid: "68749e04622047f3afbe571842e56878" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68749e04622047f3afbe571842e56878" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35475 } } }
I20260619 20:32:33.600946 13705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68749e04622047f3afbe571842e56878 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:33.600749 13704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68749e04622047f3afbe571842e56878 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68749e04622047f3afbe571842e56878" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68749e04622047f3afbe571842e56878" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35475 } } }
I20260619 20:32:33.601107 13704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68749e04622047f3afbe571842e56878 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:33.601540 13712 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:32:33.602062 13712 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:32:33.604121 13712 catalog_manager.cc:1347] Generated new cluster ID: 20aa5020ad284435a25a6e28a5eeffd7
I20260619 20:32:33.604171 13712 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:32:33.623071 13712 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:32:33.623658 13712 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:32:33.629732 13712 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68749e04622047f3afbe571842e56878: Generated new TSK 0
I20260619 20:32:33.629937 13712 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:32:33.632652 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:0
--local_ip_for_outbound_sockets=127.4.83.65
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:35475
--builtin_ntp_servers=127.4.83.84:36309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 20:32:33.714821 13722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:33.715050 13722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:33.715082 13722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:33.716357 13722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:33.716413 13722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:33.716449 13722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:32:33.717901 13722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:36309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:35475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:33.718164 13722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:33.718464 13722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:33.723977 13728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:33.724124 13727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:33.724193 13722 server_base.cc:1034] running on GCE node
W20260619 20:32:33.723938 13730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:33.724663 13722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:33.724936 13722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:33.726109 13722 hybrid_clock.cc:648] HybridClock initialized: now 1781901153726090 us; error 34 us; skew 500 ppm
I20260619 20:32:33.727427 13722 webserver.cc:460] Webserver started at http://127.4.83.65:45293/ using document root <none> and password file <none>
I20260619 20:32:33.727674 13722 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:33.727730 13722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:33.727924 13722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:33.729017 13722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/data/instance:
uuid: "0803eaca184442969e7d6596189ddb2e"
format_stamp: "Formatted at 2026-06-19 20:32:33 on dist-test-slave-4m58"
server_key: "602aa0d73145245ea210f4ba98f32d74"
server_key_iv: "df7fe4651c3170e6b3c9850ce654dba4"
server_key_version: "encryptionkey@0"
I20260619 20:32:33.729365 13722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance:
uuid: "0803eaca184442969e7d6596189ddb2e"
format_stamp: "Formatted at 2026-06-19 20:32:33 on dist-test-slave-4m58"
server_key: "602aa0d73145245ea210f4ba98f32d74"
server_key_iv: "df7fe4651c3170e6b3c9850ce654dba4"
server_key_version: "encryptionkey@0"
I20260619 20:32:33.730629 13722 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:32:33.731351 13736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:33.731488 13722 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:32:33.731573 13722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "0803eaca184442969e7d6596189ddb2e"
format_stamp: "Formatted at 2026-06-19 20:32:33 on dist-test-slave-4m58"
server_key: "602aa0d73145245ea210f4ba98f32d74"
server_key_iv: "df7fe4651c3170e6b3c9850ce654dba4"
server_key_version: "encryptionkey@0"
I20260619 20:32:33.731652 13722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:33.761890 13722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:33.762223 13722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:33.762351 13722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:33.762610 13722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:33.762976 13722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:33.763017 13722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:33.763041 13722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:33.763069 13722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:33.769749 13722 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:42933
I20260619 20:32:33.769819 13849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:42933 every 8 connection(s)
I20260619 20:32:33.770140 13722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
I20260619 20:32:33.776134 13701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:33.771016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:60733 (local address 127.4.83.126:35475)
0619 20:32:33.771101 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:33.771104 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:33.771361 (+ 257us) server_negotiation.cc:407] Connection header received
0619 20:32:33.772047 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:33.772049 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:33.772064 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:33.772120 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:33.772875 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:33.773436 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:33.774538 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:33.774750 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:33.774813 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:33.775798 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:33.775799 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:33.775800 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:32:33.775806 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:33.775822 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:32:33.775860 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:33.775908 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:33.775994 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 20:32:33.776030 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 20:32:33.776381 13852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:33.770960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35475 (local address 127.4.83.65:60733)
0619 20:32:33.771199 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:33.771243 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:33.771925 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:33.772200 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:33.772204 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:33.772215 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:33.772775 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:33.772780 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:33.773526 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:33.773527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:33.774441 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:33.774443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:33.774829 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:33.774830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:33.774978 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:33.775610 (+ 632us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:32:33.775610 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:33.775645 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:32:33.775702 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:33.775910 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:33.775916 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:33.775919 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:32:33.775989 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 20:32:33.776123 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":94,"threads_started":1}
I20260619 20:32:33.776996 13850 heartbeater.cc:347] Connected to a master server at 127.4.83.126:35475
I20260619 20:32:33.777096 13850 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:33.777336 13850 heartbeater.cc:511] Master 127.4.83.126:35475 requested a full tablet report, sending...
I20260619 20:32:33.777987 13662 ts_manager.cc:194] Registered new tserver with Master: 0803eaca184442969e7d6596189ddb2e (127.4.83.65:42933)
I20260619 20:32:33.778038 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 13722
I20260619 20:32:33.778115 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance
I20260619 20:32:33.778322 4429 external_mini_cluster.cc:1428] Setting key 4a008afd1b6f0e74883ade90b2d9075e
I20260619 20:32:33.778913 13662 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.83.65:60733
I20260619 20:32:33.779527 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:0
--local_ip_for_outbound_sockets=127.4.83.66
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:35475
--builtin_ntp_servers=127.4.83.84:36309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 20:32:33.860884 13853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:33.861094 13853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:33.861115 13853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:33.862257 13853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:33.862303 13853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:33.862329 13853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:32:33.863663 13853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:36309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:35475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:33.863924 13853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:33.864212 13853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:33.870165 13861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:33.870223 13853 server_base.cc:1034] running on GCE node
W20260619 20:32:33.870193 13859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:33.870165 13858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:33.870769 13853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:33.871104 13853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:33.872258 13853 hybrid_clock.cc:648] HybridClock initialized: now 1781901153872244 us; error 39 us; skew 500 ppm
I20260619 20:32:33.874003 13853 webserver.cc:460] Webserver started at http://127.4.83.66:38043/ using document root <none> and password file <none>
I20260619 20:32:33.874332 13853 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:33.874390 13853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:33.874543 13853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:33.876024 13853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/data/instance:
uuid: "a93d0b559c514abebe3f0f3f075baf12"
format_stamp: "Formatted at 2026-06-19 20:32:33 on dist-test-slave-4m58"
server_key: "b5ed66d79a3d1cd2e9cb10d1ebf0f96a"
server_key_iv: "12e00a64ead74726f2e489a2133e2389"
server_key_version: "encryptionkey@0"
I20260619 20:32:33.876454 13853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance:
uuid: "a93d0b559c514abebe3f0f3f075baf12"
format_stamp: "Formatted at 2026-06-19 20:32:33 on dist-test-slave-4m58"
server_key: "b5ed66d79a3d1cd2e9cb10d1ebf0f96a"
server_key_iv: "12e00a64ead74726f2e489a2133e2389"
server_key_version: "encryptionkey@0"
I20260619 20:32:33.878144 13853 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260619 20:32:33.879343 13867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:33.879675 13853 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:33.879827 13853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "a93d0b559c514abebe3f0f3f075baf12"
format_stamp: "Formatted at 2026-06-19 20:32:33 on dist-test-slave-4m58"
server_key: "b5ed66d79a3d1cd2e9cb10d1ebf0f96a"
server_key_iv: "12e00a64ead74726f2e489a2133e2389"
server_key_version: "encryptionkey@0"
I20260619 20:32:33.880046 13853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:33.899472 13853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:33.899967 13853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:33.900153 13853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:33.900456 13853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:33.900849 13853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:33.900892 13853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:33.900925 13853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:33.900949 13853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:33.908526 13853 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:34029
I20260619 20:32:33.908577 13980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:34029 every 8 connection(s)
I20260619 20:32:33.909111 13853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
I20260619 20:32:33.915102 13701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:33.910138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:60463 (local address 127.4.83.126:35475)
0619 20:32:33.910238 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:33.910241 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:33.910428 (+ 187us) server_negotiation.cc:407] Connection header received
0619 20:32:33.911060 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:33.911062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:33.911077 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:33.911133 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:33.911925 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:33.912476 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:33.913540 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:33.913750 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:33.913810 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:33.914806 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:33.914808 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:33.914809 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:32:33.914814 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:33.914830 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:32:33.914834 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:33.914877 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:33.914962 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 20:32:33.914995 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 20:32:33.915385 13983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:33.909997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35475 (local address 127.4.83.66:60463)
0619 20:32:33.910262 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:33.910312 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:33.910949 (+ 637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:33.911223 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:33.911228 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:33.911239 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:33.911816 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:33.911820 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:33.912569 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:33.912570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:33.913452 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:33.913454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:33.913825 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:33.913826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:33.913974 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:33.914647 (+ 673us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:32:33.914647 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:33.914685 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:32:33.914716 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:33.914884 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:33.914891 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:33.914895 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:32:33.914960 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 20:32:33.915121 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":116,"threads_started":1}
I20260619 20:32:33.915540 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 13853
I20260619 20:32:33.915647 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance
I20260619 20:32:33.915941 4429 external_mini_cluster.cc:1428] Setting key 9fc74cfdb01736f8c3e13afbc1dad340
I20260619 20:32:33.916024 13981 heartbeater.cc:347] Connected to a master server at 127.4.83.126:35475
I20260619 20:32:33.916193 13981 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:33.916433 13981 heartbeater.cc:511] Master 127.4.83.126:35475 requested a full tablet report, sending...
I20260619 20:32:33.916889 13662 ts_manager.cc:194] Registered new tserver with Master: a93d0b559c514abebe3f0f3f075baf12 (127.4.83.66:34029)
I20260619 20:32:33.917285 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:0
--local_ip_for_outbound_sockets=127.4.83.67
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:35475
--builtin_ntp_servers=127.4.83.84:36309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260619 20:32:33.917562 13662 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.83.66:60463
W20260619 20:32:33.997669 13984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:33.997908 13984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:33.997938 13984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:33.999135 13984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:33.999185 13984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:33.999226 13984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:32:34.000696 13984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:36309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:35475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:34.000952 13984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:34.001246 13984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:34.006727 13989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:34.006718 13992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:34.006956 13990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:34.007091 13984 server_base.cc:1034] running on GCE node
I20260619 20:32:34.007313 13984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:34.007583 13984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:34.008757 13984 hybrid_clock.cc:648] HybridClock initialized: now 1781901154008720 us; error 55 us; skew 500 ppm
I20260619 20:32:34.010401 13984 webserver.cc:460] Webserver started at http://127.4.83.67:35731/ using document root <none> and password file <none>
I20260619 20:32:34.010658 13984 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:34.010710 13984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:34.010839 13984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:34.012003 13984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/data/instance:
uuid: "b5115dbd6c3f4a2893f1f3762710631d"
format_stamp: "Formatted at 2026-06-19 20:32:34 on dist-test-slave-4m58"
server_key: "ebca4e4c8339f0e8edee868e23e3869a"
server_key_iv: "fc76c9974a3398a5fb7413ccac26fef3"
server_key_version: "encryptionkey@0"
I20260619 20:32:34.012338 13984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance:
uuid: "b5115dbd6c3f4a2893f1f3762710631d"
format_stamp: "Formatted at 2026-06-19 20:32:34 on dist-test-slave-4m58"
server_key: "ebca4e4c8339f0e8edee868e23e3869a"
server_key_iv: "fc76c9974a3398a5fb7413ccac26fef3"
server_key_version: "encryptionkey@0"
I20260619 20:32:34.013779 13984 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:34.014770 13998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:34.014997 13984 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:34.015098 13984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "b5115dbd6c3f4a2893f1f3762710631d"
format_stamp: "Formatted at 2026-06-19 20:32:34 on dist-test-slave-4m58"
server_key: "ebca4e4c8339f0e8edee868e23e3869a"
server_key_iv: "fc76c9974a3398a5fb7413ccac26fef3"
server_key_version: "encryptionkey@0"
I20260619 20:32:34.015166 13984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:34.025676 13984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:34.026031 13984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:34.026162 13984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:34.026386 13984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:34.026727 13984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:34.026764 13984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:34.026795 13984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:34.026813 13984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:34.034027 13984 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:38109
I20260619 20:32:34.034132 14111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:38109 every 8 connection(s)
I20260619 20:32:34.034482 13984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
I20260619 20:32:34.040745 13701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:34.035467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:51327 (local address 127.4.83.126:35475)
0619 20:32:34.035550 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:34.035553 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:34.035967 (+ 414us) server_negotiation.cc:407] Connection header received
0619 20:32:34.036537 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:34.036539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:34.036551 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:34.036620 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:34.037371 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:34.037957 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:34.039058 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:34.039304 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:34.039392 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:34.040385 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:34.040387 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:34.040387 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:32:34.040394 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:34.040416 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:32:34.040421 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:34.040479 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:34.040616 (+ 137us) server_negotiation.cc:299] Negotiation successful
0619 20:32:34.040653 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 20:32:34.040942 14114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:34.035458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35475 (local address 127.4.83.67:51327)
0619 20:32:34.035747 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:34.035812 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:34.036392 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:34.036685 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:34.036689 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:34.036698 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:34.037230 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:34.037234 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:34.038083 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:34.038084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:34.038931 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:34.038933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:34.039408 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:34.039409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:34.039537 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:34.040211 (+ 674us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:32:34.040212 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:34.040241 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:32:34.040268 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:34.040518 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:34.040526 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:34.040529 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:32:34.040611 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 20:32:34.040705 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":132,"threads_started":1}
I20260619 20:32:34.041647 14112 heartbeater.cc:347] Connected to a master server at 127.4.83.126:35475
I20260619 20:32:34.041775 14112 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:34.041982 14112 heartbeater.cc:511] Master 127.4.83.126:35475 requested a full tablet report, sending...
I20260619 20:32:34.042455 13662 ts_manager.cc:194] Registered new tserver with Master: b5115dbd6c3f4a2893f1f3762710631d (127.4.83.67:38109)
I20260619 20:32:34.042853 13662 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.83.67:51327
I20260619 20:32:34.043532 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 13984
I20260619 20:32:34.043745 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance
I20260619 20:32:34.044095 4429 external_mini_cluster.cc:1428] Setting key c1e06466a913dac2c7c4aca409c9acb0
I20260619 20:32:34.045079 4429 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 20:32:34.047410 13701 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:34.046741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59976 (local address 127.4.83.126:35475)
0619 20:32:34.046805 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:34.046808 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:34.046986 (+ 178us) server_negotiation.cc:407] Connection header received
0619 20:32:34.047056 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:34.047057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:34.047070 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:34.047118 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:34.047322 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59976: BlockingRecv error: recv got EOF from 127.0.0.1:59976 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 20:32:34.047473 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 13722
I20260619 20:32:34.052672 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 13853
I20260619 20:32:34.057487 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 13984
I20260619 20:32:34.062601 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 13631
2026-06-19T20:32:34Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (644 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-19T20:32:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:32:34Z Disabled control of system clock
I20260619 20:32:34.091454 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:46621
--webserver_interface=127.4.83.126
--webserver_port=0
--builtin_ntp_servers=127.4.83.84:34003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:46621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 20:32:34.184597 14123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:34.184909 14123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:34.184937 14123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:34.186378 14123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:32:34.186443 14123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:34.186461 14123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:34.186483 14123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:32:34.186499 14123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:32:34.188220 14123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:34003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:46621
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:46621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:34.188512 14123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:34.188853 14123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:34.194444 14128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:34.194516 14123 server_base.cc:1034] running on GCE node
W20260619 20:32:34.194461 14131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:34.194446 14129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:34.195039 14123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:34.195454 14123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:34.196623 14123 hybrid_clock.cc:648] HybridClock initialized: now 1781901154196597 us; error 75 us; skew 500 ppm
I20260619 20:32:34.202508 14123 webserver.cc:460] Webserver started at http://127.4.83.126:34939/ using document root <none> and password file <none>
I20260619 20:32:34.202819 14123 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:34.202879 14123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:34.203030 14123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:34.204250 14123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "e30b13b7139b476a9cdceea1cec6fa83"
format_stamp: "Formatted at 2026-06-19 20:32:34 on dist-test-slave-4m58"
server_key: "c7ad5d54e830a4ee1671c12f6bd8ec0b"
server_key_iv: "d4b27591971bd4bf41c133226d9b1dba"
server_key_version: "encryptionkey@0"
I20260619 20:32:34.204615 14123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "e30b13b7139b476a9cdceea1cec6fa83"
format_stamp: "Formatted at 2026-06-19 20:32:34 on dist-test-slave-4m58"
server_key: "c7ad5d54e830a4ee1671c12f6bd8ec0b"
server_key_iv: "d4b27591971bd4bf41c133226d9b1dba"
server_key_version: "encryptionkey@0"
I20260619 20:32:34.206162 14123 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:32:34.207034 14137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:34.207229 14123 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:34.207331 14123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "e30b13b7139b476a9cdceea1cec6fa83"
format_stamp: "Formatted at 2026-06-19 20:32:34 on dist-test-slave-4m58"
server_key: "c7ad5d54e830a4ee1671c12f6bd8ec0b"
server_key_iv: "d4b27591971bd4bf41c133226d9b1dba"
server_key_version: "encryptionkey@0"
I20260619 20:32:34.207413 14123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:34.210839 14123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:34.211126 14123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:34.211252 14123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:34.215250 14123 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:46621
I20260619 20:32:34.215315 14189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:46621 every 8 connection(s)
I20260619 20:32:34.215651 14123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:32:34.216472 14190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:34.217147 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 14123
I20260619 20:32:34.217240 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:32:34.217434 4429 external_mini_cluster.cc:1428] Setting key ed87777ec21a8ec43c5beb0541f2c621
I20260619 20:32:34.220541 14193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:34.218017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57060 (local address 127.4.83.126:46621)
0619 20:32:34.218295 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:34.218307 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:34.218342 (+ 35us) server_negotiation.cc:407] Connection header received
0619 20:32:34.218867 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:34.218878 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:34.218943 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:34.219137 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:34.219951 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:34.219955 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:34.219956 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:32:34.219961 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:34.220015 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:32:34.220030 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:34.220118 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:34.220238 (+ 120us) server_negotiation.cc:299] Negotiation successful
0619 20:32:34.220308 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":79,"threads_started":1}
I20260619 20:32:34.221066 14190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e30b13b7139b476a9cdceea1cec6fa83: Bootstrap starting.
I20260619 20:32:34.221800 14190 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e30b13b7139b476a9cdceea1cec6fa83: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:34.222054 14190 log.cc:824] T 00000000000000000000000000000000 P e30b13b7139b476a9cdceea1cec6fa83: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:34.222774 14190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e30b13b7139b476a9cdceea1cec6fa83: No bootstrap required, opened a new log
I20260619 20:32:34.224311 14190 raft_consensus.cc:348] T 00000000000000000000000000000000 P e30b13b7139b476a9cdceea1cec6fa83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e30b13b7139b476a9cdceea1cec6fa83" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 46621 } }
I20260619 20:32:34.224474 14190 raft_consensus.cc:374] T 00000000000000000000000000000000 P e30b13b7139b476a9cdceea1cec6fa83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:34.224498 14190 raft_consensus.cc:729] T 00000000000000000000000000000000 P e30b13b7139b476a9cdceea1cec6fa83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e30b13b7139b476a9cdceea1cec6fa83, State: Initialized, Role: FOLLOWER
I20260619 20:32:34.224642 14190 consensus_queue.cc:260] T 00000000000000000000000000000000 P e30b13b7139b476a9cdceea1cec6fa83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e30b13b7139b476a9cdceea1cec6fa83" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 46621 } }
I20260619 20:32:34.224731 14190 raft_consensus.cc:388] T 00000000000000000000000000000000 P e30b13b7139b476a9cdceea1cec6fa83 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:32:34.224754 14190 raft_consensus.cc:482] T 00000000000000000000000000000000 P e30b13b7139b476a9cdceea1cec6fa83 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:32:34.224794 14190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e30b13b7139b476a9cdceea1cec6fa83 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:34.225818 14190 raft_consensus.cc:504] T 00000000000000000000000000000000 P e30b13b7139b476a9cdceea1cec6fa83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e30b13b7139b476a9cdceea1cec6fa83" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 46621 } }
I20260619 20:32:34.225943 14190 leader_election.cc:302] T 00000000000000000000000000000000 P e30b13b7139b476a9cdceea1cec6fa83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e30b13b7139b476a9cdceea1cec6fa83; no voters:
I20260619 20:32:34.226150 14190 leader_election.cc:288] T 00000000000000000000000000000000 P e30b13b7139b476a9cdceea1cec6fa83 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:32:34.226281 14195 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e30b13b7139b476a9cdceea1cec6fa83 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:32:34.226577 14195 raft_consensus.cc:686] T 00000000000000000000000000000000 P e30b13b7139b476a9cdceea1cec6fa83 [term 1 LEADER]: Becoming Leader. State: Replica: e30b13b7139b476a9cdceea1cec6fa83, State: Running, Role: LEADER
I20260619 20:32:34.226639 14190 sys_catalog.cc:564] T 00000000000000000000000000000000 P e30b13b7139b476a9cdceea1cec6fa83 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:32:34.226716 14195 consensus_queue.cc:237] T 00000000000000000000000000000000 P e30b13b7139b476a9cdceea1cec6fa83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e30b13b7139b476a9cdceea1cec6fa83" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 46621 } }
I20260619 20:32:34.227208 14196 sys_catalog.cc:455] T 00000000000000000000000000000000 P e30b13b7139b476a9cdceea1cec6fa83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e30b13b7139b476a9cdceea1cec6fa83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e30b13b7139b476a9cdceea1cec6fa83" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 46621 } } }
I20260619 20:32:34.227170 14197 sys_catalog.cc:455] T 00000000000000000000000000000000 P e30b13b7139b476a9cdceea1cec6fa83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e30b13b7139b476a9cdceea1cec6fa83. Latest consensus state: current_term: 1 leader_uuid: "e30b13b7139b476a9cdceea1cec6fa83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e30b13b7139b476a9cdceea1cec6fa83" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 46621 } } }
I20260619 20:32:34.227339 14196 sys_catalog.cc:458] T 00000000000000000000000000000000 P e30b13b7139b476a9cdceea1cec6fa83 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:34.227366 14197 sys_catalog.cc:458] T 00000000000000000000000000000000 P e30b13b7139b476a9cdceea1cec6fa83 [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:34.227579 14204 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:32:34.228152 14204 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:32:34.229741 14204 catalog_manager.cc:1347] Generated new cluster ID: 4ff771f46817480990370da58e06eec6
I20260619 20:32:34.229796 14204 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:32:34.256035 14204 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:32:34.256237 14204 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:32:34.266294 14204 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e30b13b7139b476a9cdceea1cec6fa83: Generated new TSK 0
I20260619 20:32:34.266511 14204 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:32:34.269752 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:0
--local_ip_for_outbound_sockets=127.4.83.65
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:46621
--builtin_ntp_servers=127.4.83.84:34003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 20:32:34.355996 14214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:34.356209 14214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:34.356232 14214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:34.357385 14214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:34.357421 14214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:34.357445 14214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:32:34.358752 14214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:34003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:46621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:34.358958 14214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:34.359232 14214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:34.364107 14222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:34.364096 14220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:34.364163 14214 server_base.cc:1034] running on GCE node
W20260619 20:32:34.364096 14219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:34.364538 14214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:34.364743 14214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:34.365899 14214 hybrid_clock.cc:648] HybridClock initialized: now 1781901154365884 us; error 32 us; skew 500 ppm
I20260619 20:32:34.367146 14214 webserver.cc:460] Webserver started at http://127.4.83.65:41411/ using document root <none> and password file <none>
I20260619 20:32:34.367394 14214 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:34.367444 14214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:34.367566 14214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:34.368633 14214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/data/instance:
uuid: "f160859320dd49729924dffa0ffcb0e8"
format_stamp: "Formatted at 2026-06-19 20:32:34 on dist-test-slave-4m58"
server_key: "9534d37d37343cd84d869339793c7937"
server_key_iv: "3ab76c1d0688e0fe4cbc1002d7a23a8b"
server_key_version: "encryptionkey@0"
I20260619 20:32:34.368948 14214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance:
uuid: "f160859320dd49729924dffa0ffcb0e8"
format_stamp: "Formatted at 2026-06-19 20:32:34 on dist-test-slave-4m58"
server_key: "9534d37d37343cd84d869339793c7937"
server_key_iv: "3ab76c1d0688e0fe4cbc1002d7a23a8b"
server_key_version: "encryptionkey@0"
I20260619 20:32:34.370111 14214 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:32:34.370859 14228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:34.371021 14214 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:32:34.371106 14214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "f160859320dd49729924dffa0ffcb0e8"
format_stamp: "Formatted at 2026-06-19 20:32:34 on dist-test-slave-4m58"
server_key: "9534d37d37343cd84d869339793c7937"
server_key_iv: "3ab76c1d0688e0fe4cbc1002d7a23a8b"
server_key_version: "encryptionkey@0"
I20260619 20:32:34.371168 14214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:34.374430 14214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:34.374656 14214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:34.374765 14214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:34.374940 14214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:34.375206 14214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:34.375236 14214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:34.375272 14214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:34.375295 14214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:34.380712 14214 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:42237
I20260619 20:32:34.380776 14341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:42237 every 8 connection(s)
I20260619 20:32:34.381070 14214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
I20260619 20:32:34.383984 14193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:34.381786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:60635 (local address 127.4.83.126:46621)
0619 20:32:34.381848 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:34.381850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:34.382005 (+ 155us) server_negotiation.cc:407] Connection header received
0619 20:32:34.382472 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:34.382473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:34.382478 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:34.382523 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:34.383622 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:34.383623 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:34.383623 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:32:34.383627 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:34.383641 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:32:34.383645 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:34.383692 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:34.383913 (+ 221us) server_negotiation.cc:299] Negotiation successful
0619 20:32:34.383928 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 20:32:34.384086 14344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:34.381719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:46621 (local address 127.4.83.65:60635)
0619 20:32:34.381894 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:34.381930 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:34.382393 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:34.382579 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:34.382582 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:34.382590 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:34.383489 (+ 899us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:32:34.383490 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:34.383525 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:32:34.383553 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:34.383730 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:34.383736 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:34.383738 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:32:34.383813 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 20:32:34.383905 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":69,"threads_started":1}
I20260619 20:32:34.384503 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 14214
I20260619 20:32:34.384541 14342 heartbeater.cc:347] Connected to a master server at 127.4.83.126:46621
I20260619 20:32:34.384601 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance
I20260619 20:32:34.384624 14342 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:34.384775 14342 heartbeater.cc:511] Master 127.4.83.126:46621 requested a full tablet report, sending...
I20260619 20:32:34.384800 4429 external_mini_cluster.cc:1428] Setting key bf1ef9571d1e16f267acb9135316531d
I20260619 20:32:34.385388 14153 ts_manager.cc:194] Registered new tserver with Master: f160859320dd49729924dffa0ffcb0e8 (127.4.83.65:42237)
I20260619 20:32:34.385787 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:0
--local_ip_for_outbound_sockets=127.4.83.66
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:46621
--builtin_ntp_servers=127.4.83.84:34003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 20:32:34.465837 14345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:34.466080 14345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:34.466104 14345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:34.467326 14345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:34.467365 14345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:34.467393 14345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:32:34.468777 14345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:34003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:46621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:34.468982 14345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:34.469237 14345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:34.474337 14350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:34.474442 14345 server_base.cc:1034] running on GCE node
W20260619 20:32:34.474334 14351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:34.474593 14353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:34.474814 14345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:34.475093 14345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:34.476262 14345 hybrid_clock.cc:648] HybridClock initialized: now 1781901154476227 us; error 50 us; skew 500 ppm
I20260619 20:32:34.477635 14345 webserver.cc:460] Webserver started at http://127.4.83.66:40963/ using document root <none> and password file <none>
I20260619 20:32:34.477900 14345 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:34.477952 14345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:34.478084 14345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:34.479182 14345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/data/instance:
uuid: "81663e2837d1420c91f7a0bce6da8644"
format_stamp: "Formatted at 2026-06-19 20:32:34 on dist-test-slave-4m58"
server_key: "a3d1cbf0bc73e416aaf0edee6480dc0d"
server_key_iv: "b5a8af72d3dd7f633dba35395c5c7843"
server_key_version: "encryptionkey@0"
I20260619 20:32:34.479533 14345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance:
uuid: "81663e2837d1420c91f7a0bce6da8644"
format_stamp: "Formatted at 2026-06-19 20:32:34 on dist-test-slave-4m58"
server_key: "a3d1cbf0bc73e416aaf0edee6480dc0d"
server_key_iv: "b5a8af72d3dd7f633dba35395c5c7843"
server_key_version: "encryptionkey@0"
I20260619 20:32:34.480881 14345 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:32:34.481649 14359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:34.481812 14345 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:32:34.481891 14345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "81663e2837d1420c91f7a0bce6da8644"
format_stamp: "Formatted at 2026-06-19 20:32:34 on dist-test-slave-4m58"
server_key: "a3d1cbf0bc73e416aaf0edee6480dc0d"
server_key_iv: "b5a8af72d3dd7f633dba35395c5c7843"
server_key_version: "encryptionkey@0"
I20260619 20:32:34.481958 14345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:34.485909 14345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:34.486195 14345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:34.486318 14345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:34.486510 14345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:34.486829 14345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:34.486861 14345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:34.486889 14345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:34.486913 14345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:34.493207 14345 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:33013
I20260619 20:32:34.493265 14472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:33013 every 8 connection(s)
I20260619 20:32:34.493595 14345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
I20260619 20:32:34.496896 14193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:34.494427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:47769 (local address 127.4.83.126:46621)
0619 20:32:34.494481 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:34.494483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:34.494721 (+ 238us) server_negotiation.cc:407] Connection header received
0619 20:32:34.495248 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:34.495250 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:34.495254 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:34.495316 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:34.496665 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:34.496666 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:34.496667 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:32:34.496671 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:34.496688 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:32:34.496692 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:34.496742 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:34.496818 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 20:32:34.496833 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 20:32:34.497094 14475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:34.494362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:46621 (local address 127.4.83.66:47769)
0619 20:32:34.494587 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:34.494627 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:34.495098 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:34.495424 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:34.495428 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:34.495437 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:34.496487 (+ 1050us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:32:34.496487 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:34.496519 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:32:34.496552 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:34.496750 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:34.496753 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:34.496756 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:32:34.496811 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 20:32:34.496881 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":83,"threads_started":1}
I20260619 20:32:34.497660 14473 heartbeater.cc:347] Connected to a master server at 127.4.83.126:46621
I20260619 20:32:34.497756 14473 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:34.497927 14473 heartbeater.cc:511] Master 127.4.83.126:46621 requested a full tablet report, sending...
I20260619 20:32:34.498294 14153 ts_manager.cc:194] Registered new tserver with Master: 81663e2837d1420c91f7a0bce6da8644 (127.4.83.66:33013)
I20260619 20:32:34.500399 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 14345
I20260619 20:32:34.500485 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance
I20260619 20:32:34.500663 4429 external_mini_cluster.cc:1428] Setting key 89fbe1da9659ce3c80dac7c44eaaf627
I20260619 20:32:34.501861 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:0
--local_ip_for_outbound_sockets=127.4.83.67
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:46621
--builtin_ntp_servers=127.4.83.84:34003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 20:32:34.584039 14476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:34.584278 14476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:34.584313 14476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:34.585613 14476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:34.585664 14476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:34.585702 14476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:32:34.587172 14476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:34003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:46621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:34.587435 14476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:34.587738 14476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:32:34.593093 14476 server_base.cc:1034] running on GCE node
W20260619 20:32:34.593062 14484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:34.593061 14481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:34.593056 14482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:34.593487 14476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:34.593729 14476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:34.594864 14476 hybrid_clock.cc:648] HybridClock initialized: now 1781901154594855 us; error 30 us; skew 500 ppm
I20260619 20:32:34.596158 14476 webserver.cc:460] Webserver started at http://127.4.83.67:36583/ using document root <none> and password file <none>
I20260619 20:32:34.596386 14476 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:34.596431 14476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:34.596553 14476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:34.597641 14476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/data/instance:
uuid: "ea2dfab4d23a4a37b6a19814ce361419"
format_stamp: "Formatted at 2026-06-19 20:32:34 on dist-test-slave-4m58"
server_key: "044de905d1397f2a3547032403122f9b"
server_key_iv: "d5f02683e60d30a1525534796e7721c5"
server_key_version: "encryptionkey@0"
I20260619 20:32:34.597980 14476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance:
uuid: "ea2dfab4d23a4a37b6a19814ce361419"
format_stamp: "Formatted at 2026-06-19 20:32:34 on dist-test-slave-4m58"
server_key: "044de905d1397f2a3547032403122f9b"
server_key_iv: "d5f02683e60d30a1525534796e7721c5"
server_key_version: "encryptionkey@0"
I20260619 20:32:34.599247 14476 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:32:34.599987 14490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:34.600126 14476 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:32:34.600205 14476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "ea2dfab4d23a4a37b6a19814ce361419"
format_stamp: "Formatted at 2026-06-19 20:32:34 on dist-test-slave-4m58"
server_key: "044de905d1397f2a3547032403122f9b"
server_key_iv: "d5f02683e60d30a1525534796e7721c5"
server_key_version: "encryptionkey@0"
I20260619 20:32:34.600273 14476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:34.603627 14476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:34.603950 14476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:34.604074 14476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:34.604256 14476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:34.604552 14476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:34.604584 14476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:34.604612 14476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:34.604635 14476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:34.610544 14476 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:46481
I20260619 20:32:34.610584 14603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:46481 every 8 connection(s)
I20260619 20:32:34.610935 14476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
I20260619 20:32:34.614051 14193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:34.611667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:33313 (local address 127.4.83.126:46621)
0619 20:32:34.611733 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:34.611735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:34.611968 (+ 233us) server_negotiation.cc:407] Connection header received
0619 20:32:34.612435 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:34.612436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:34.612442 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:34.612504 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:34.613623 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:34.613624 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:34.613625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:32:34.613638 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:34.613653 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:32:34.613658 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:34.613722 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:34.613924 (+ 202us) server_negotiation.cc:299] Negotiation successful
0619 20:32:34.613938 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 20:32:34.614128 14606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:34.611633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:46621 (local address 127.4.83.67:33313)
0619 20:32:34.611821 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:34.611887 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:34.612352 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:34.612556 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:34.612559 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:34.612567 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:34.613461 (+ 894us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:32:34.613461 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:34.613491 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:32:34.613522 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:34.613796 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:34.613802 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:34.613805 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:32:34.613886 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 20:32:34.613943 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
I20260619 20:32:34.614570 14604 heartbeater.cc:347] Connected to a master server at 127.4.83.126:46621
I20260619 20:32:34.614660 14604 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:34.614804 14604 heartbeater.cc:511] Master 127.4.83.126:46621 requested a full tablet report, sending...
I20260619 20:32:34.615128 14153 ts_manager.cc:194] Registered new tserver with Master: ea2dfab4d23a4a37b6a19814ce361419 (127.4.83.67:46481)
I20260619 20:32:34.616858 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 14476
I20260619 20:32:34.616948 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance
I20260619 20:32:34.617095 4429 external_mini_cluster.cc:1428] Setting key 2e67c32ffb1355001f6d290e293805b1
I20260619 20:32:34.617859 4429 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 20:32:34.619652 14193 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:32:34.619192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57070 (local address 127.4.83.126:46621)
0619 20:32:34.619255 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:34.619258 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:34.619350 (+ 92us) server_negotiation.cc:407] Connection header received
0619 20:32:34.619413 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:34.619414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:34.619419 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:34.619464 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:34.619583 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57070: BlockingRecv error: recv got EOF from 127.0.0.1:57070 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 20:32:34.619680 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 14214
I20260619 20:32:34.624670 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 14345
I20260619 20:32:34.628614 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 14476
I20260619 20:32:34.633072 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 14123
2026-06-19T20:32:34Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (568 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:32:34 dist-test-slave-4m58 krb5kdc[14614](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:32:34 dist-test-slave-4m58 krb5kdc[14614](info): set up 2 sockets
Jun 19 20:32:34 dist-test-slave-4m58 krb5kdc[14614](info): commencing operation
krb5kdc: starting...
W20260619 20:32:35.601701 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.950s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:32:35 dist-test-slave-4m58 krb5kdc[14614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:32:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:32:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
I20260619 20:32:35.718478 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:40261
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:35633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:40261
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:35.797406 14627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:35.797647 14627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:35.797679 14627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:35.798893 14627 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:32:35.798951 14627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:35.798970 14627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:35.798982 14627 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:32:35.798993 14627 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:32:35.800573 14627 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:35633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:40261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:40261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:35.800791 14627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:35.800982 14627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:35.805866 14633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:35.805850 14635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:35.805840 14632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:35.806231 14627 server_base.cc:1034] running on GCE node
I20260619 20:32:35.806464 14627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:35.806749 14627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:35.807883 14627 hybrid_clock.cc:648] HybridClock initialized: now 1781901155807869 us; error 32 us; skew 500 ppm
Jun 19 20:32:35 dist-test-slave-4m58 krb5kdc[14614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901155, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:35.810328 14627 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:32:35.810737 14627 webserver.cc:460] Webserver started at http://127.4.83.126:42879/ using document root <none> and password file <none>
I20260619 20:32:35.810938 14627 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:35.810985 14627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:35.811107 14627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:35.812299 14627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "402047a7c26f4f009325fda6251bc65f"
format_stamp: "Formatted at 2026-06-19 20:32:35 on dist-test-slave-4m58"
server_key: "69417debb11e8df4c819e3544a901ddf"
server_key_iv: "0a5fb0648837c6ef14cea89047a2b0bb"
server_key_version: "encryptionkey@0"
I20260619 20:32:35.812649 14627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "402047a7c26f4f009325fda6251bc65f"
format_stamp: "Formatted at 2026-06-19 20:32:35 on dist-test-slave-4m58"
server_key: "69417debb11e8df4c819e3544a901ddf"
server_key_iv: "0a5fb0648837c6ef14cea89047a2b0bb"
server_key_version: "encryptionkey@0"
I20260619 20:32:35.813913 14627 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:32:35.814575 14642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:35.814756 14627 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:32:35.814846 14627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "402047a7c26f4f009325fda6251bc65f"
format_stamp: "Formatted at 2026-06-19 20:32:35 on dist-test-slave-4m58"
server_key: "69417debb11e8df4c819e3544a901ddf"
server_key_iv: "0a5fb0648837c6ef14cea89047a2b0bb"
server_key_version: "encryptionkey@0"
I20260619 20:32:35.814905 14627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:35.829090 14627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:35.831171 14627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:35.831282 14627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:35.834997 14627 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:40261
I20260619 20:32:35.835031 14694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:40261 every 8 connection(s)
I20260619 20:32:35.835346 14627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:32:35.836323 14695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:35.838553 14695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 402047a7c26f4f009325fda6251bc65f: Bootstrap starting.
I20260619 20:32:35.839062 14695 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 402047a7c26f4f009325fda6251bc65f: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:35.839278 14695 log.cc:824] T 00000000000000000000000000000000 P 402047a7c26f4f009325fda6251bc65f: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:35.839931 14695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 402047a7c26f4f009325fda6251bc65f: No bootstrap required, opened a new log
I20260619 20:32:35.841497 14695 raft_consensus.cc:348] T 00000000000000000000000000000000 P 402047a7c26f4f009325fda6251bc65f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "402047a7c26f4f009325fda6251bc65f" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 40261 } }
I20260619 20:32:35.841635 14695 raft_consensus.cc:374] T 00000000000000000000000000000000 P 402047a7c26f4f009325fda6251bc65f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:35.841656 14695 raft_consensus.cc:729] T 00000000000000000000000000000000 P 402047a7c26f4f009325fda6251bc65f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 402047a7c26f4f009325fda6251bc65f, State: Initialized, Role: FOLLOWER
I20260619 20:32:35.841753 14695 consensus_queue.cc:260] T 00000000000000000000000000000000 P 402047a7c26f4f009325fda6251bc65f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "402047a7c26f4f009325fda6251bc65f" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 40261 } }
I20260619 20:32:35.841804 14695 raft_consensus.cc:388] T 00000000000000000000000000000000 P 402047a7c26f4f009325fda6251bc65f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:32:35.841830 14695 raft_consensus.cc:482] T 00000000000000000000000000000000 P 402047a7c26f4f009325fda6251bc65f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:32:35.841868 14695 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 402047a7c26f4f009325fda6251bc65f [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:35.842634 14695 raft_consensus.cc:504] T 00000000000000000000000000000000 P 402047a7c26f4f009325fda6251bc65f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "402047a7c26f4f009325fda6251bc65f" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 40261 } }
I20260619 20:32:35.842728 14695 leader_election.cc:302] T 00000000000000000000000000000000 P 402047a7c26f4f009325fda6251bc65f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 402047a7c26f4f009325fda6251bc65f; no voters:
I20260619 20:32:35.842887 14695 leader_election.cc:288] T 00000000000000000000000000000000 P 402047a7c26f4f009325fda6251bc65f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:32:35.842935 14698 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 402047a7c26f4f009325fda6251bc65f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:32:35.843084 14698 raft_consensus.cc:686] T 00000000000000000000000000000000 P 402047a7c26f4f009325fda6251bc65f [term 1 LEADER]: Becoming Leader. State: Replica: 402047a7c26f4f009325fda6251bc65f, State: Running, Role: LEADER
I20260619 20:32:35.843134 14695 sys_catalog.cc:564] T 00000000000000000000000000000000 P 402047a7c26f4f009325fda6251bc65f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:32:35.843221 14698 consensus_queue.cc:237] T 00000000000000000000000000000000 P 402047a7c26f4f009325fda6251bc65f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "402047a7c26f4f009325fda6251bc65f" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 40261 } }
I20260619 20:32:35.843416 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 14627
I20260619 20:32:35.843514 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:32:35.843734 4429 external_mini_cluster.cc:1428] Setting key 436b57c19b34a7dee233c97e60ba37f5
I20260619 20:32:35.843726 14700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 402047a7c26f4f009325fda6251bc65f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 402047a7c26f4f009325fda6251bc65f. Latest consensus state: current_term: 1 leader_uuid: "402047a7c26f4f009325fda6251bc65f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "402047a7c26f4f009325fda6251bc65f" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 40261 } } }
I20260619 20:32:35.843792 14700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 402047a7c26f4f009325fda6251bc65f [sys.catalog]: This master's current role is: LEADER
I20260619 20:32:35.844605 14699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 402047a7c26f4f009325fda6251bc65f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "402047a7c26f4f009325fda6251bc65f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "402047a7c26f4f009325fda6251bc65f" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 40261 } } }
I20260619 20:32:35.844691 14699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 402047a7c26f4f009325fda6251bc65f [sys.catalog]: This master's current role is: LEADER
W20260619 20:32:35.844899 14715 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 402047a7c26f4f009325fda6251bc65f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:32:35.844944 14715 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:32:35.844990 14707 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:32:35.845183 14707 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:32:35.846696 14707 catalog_manager.cc:1347] Generated new cluster ID: cc2bf69a5b9247869877ab0976325b6e
I20260619 20:32:35.846748 14707 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:32:35 dist-test-slave-4m58 krb5kdc[14614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:32:35.854178 14716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:35.844428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54096 (local address 127.4.83.126:40261)
0619 20:32:35.844635 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:35.844647 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:35.844681 (+ 34us) server_negotiation.cc:407] Connection header received
0619 20:32:35.844805 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:35.844814 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:35.844885 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:35.845028 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:35.845839 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:35.847058 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:35.847987 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:35.848475 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:35.848583 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:35.851012 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:35.851017 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:35.851017 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:35.851031 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:35.852658 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:35.852978 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:35.852980 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:35.852981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:35.853080 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:35.853252 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:35.853253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:35.853253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:35.853517 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:35.853660 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:35.853771 (+ 111us) server_negotiation.cc:299] Negotiation successful
0619 20:32:35.854002 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":67,"threads_started":1}
I20260619 20:32:35.872298 14707 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:32:35.873101 14707 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:32:35.887265 14707 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 402047a7c26f4f009325fda6251bc65f: Generated new TSK 0
I20260619 20:32:35.887508 14707 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.83.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
I20260619 20:32:35.926482 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:0
--local_ip_for_outbound_sockets=127.4.83.65
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:40261
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:35633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:36.038036 14721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:36.038280 14721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:36.038316 14721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:36.039541 14721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:36.039590 14721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:36.039629 14721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:32:36.041193 14721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:35633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:40261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:36.041450 14721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:36.041683 14721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:36.046890 14727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:36.046882 14726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:36.046907 14729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:36.047184 14721 server_base.cc:1034] running on GCE node
I20260619 20:32:36.047392 14721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:36.047636 14721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:36.048782 14721 hybrid_clock.cc:648] HybridClock initialized: now 1781901156048768 us; error 33 us; skew 500 ppm
Jun 19 20:32:36 dist-test-slave-4m58 krb5kdc[14614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901156, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:36.051265 14721 init.cc:377] Logged in from keytab as kudu/127.4.83.65@KRBTEST.COM (short username kudu)
I20260619 20:32:36.051766 14721 webserver.cc:460] Webserver started at http://127.4.83.65:46081/ using document root <none> and password file <none>
I20260619 20:32:36.052014 14721 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:36.052060 14721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:36.052167 14721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:36.053232 14721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/data/instance:
uuid: "068d492b6b88455fbe366c0e277e2bde"
format_stamp: "Formatted at 2026-06-19 20:32:36 on dist-test-slave-4m58"
server_key: "440a9a65d9b3b946c5b9b45ee1340d6c"
server_key_iv: "46e6f741fea6cd9e0ce8866d1267b05e"
server_key_version: "encryptionkey@0"
I20260619 20:32:36.053697 14721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance:
uuid: "068d492b6b88455fbe366c0e277e2bde"
format_stamp: "Formatted at 2026-06-19 20:32:36 on dist-test-slave-4m58"
server_key: "440a9a65d9b3b946c5b9b45ee1340d6c"
server_key_iv: "46e6f741fea6cd9e0ce8866d1267b05e"
server_key_version: "encryptionkey@0"
I20260619 20:32:36.055375 14721 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:32:36.056186 14736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:36.056381 14721 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:32:36.056478 14721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "068d492b6b88455fbe366c0e277e2bde"
format_stamp: "Formatted at 2026-06-19 20:32:36 on dist-test-slave-4m58"
server_key: "440a9a65d9b3b946c5b9b45ee1340d6c"
server_key_iv: "46e6f741fea6cd9e0ce8866d1267b05e"
server_key_version: "encryptionkey@0"
I20260619 20:32:36.056547 14721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:36.069104 14721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:36.071408 14721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:36.071532 14721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:36.071722 14721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:36.072088 14721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:36.072126 14721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:36.072151 14721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:36.072175 14721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:36.077631 14721 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:34981
I20260619 20:32:36.077678 14849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:34981 every 8 connection(s)
I20260619 20:32:36.077980 14721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
I20260619 20:32:36.082367 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 14721
I20260619 20:32:36.082479 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance
I20260619 20:32:36.082679 4429 external_mini_cluster.cc:1428] Setting key 6e20b04ff399936cef939e74cb1e2746
Jun 19 20:32:36 dist-test-slave-4m58 krb5kdc[14614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901156, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:32:36.087895 14716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:36.078712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:43887 (local address 127.4.83.126:40261)
0619 20:32:36.078805 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:36.078807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:36.078988 (+ 181us) server_negotiation.cc:407] Connection header received
0619 20:32:36.079472 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:36.079473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:36.079483 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:36.079529 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:36.080568 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.081081 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.082104 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.082354 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.082410 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:36.084947 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:36.084949 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:36.084949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:36.084958 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:36.086466 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:36.086882 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:36.086884 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:36.086884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:36.086950 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:36.087186 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:36.087187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:36.087187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:36.087326 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:36.087391 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:36.087646 (+ 255us) server_negotiation.cc:299] Negotiation successful
0619 20:32:36.087754 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 20:32:36.087977 14852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:36.078629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:40261 (local address 127.4.83.65:43887)
0619 20:32:36.078871 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:36.078908 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:36.079382 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:36.079589 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:36.079592 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:36.080236 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:36.080448 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:36.080454 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.081179 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.081180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:36.082029 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:36.082031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.082458 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.082459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:36.082616 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:36.083153 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:36.083161 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:36.084836 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:36.086553 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:36.086561 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:36.086563 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:36.086804 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:36.087014 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:36.087014 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:36.087014 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:36.087126 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:36.087389 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:36.087393 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:36.087519 (+ 126us) client_negotiation.cc:769] Sending connection context
0619 20:32:36.087597 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 20:32:36.087727 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":93,"threads_started":1}
I20260619 20:32:36.088485 14850 heartbeater.cc:347] Connected to a master server at 127.4.83.126:40261
I20260619 20:32:36.088593 14850 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:36.088773 14850 heartbeater.cc:511] Master 127.4.83.126:40261 requested a full tablet report, sending...
I20260619 20:32:36.089336 14659 ts_manager.cc:194] Registered new tserver with Master: 068d492b6b88455fbe366c0e277e2bde (127.4.83.65:34981)
I20260619 20:32:36.090232 14659 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.65@KRBTEST.COM'} at 127.4.83.65:43887
WARNING: no policy specified for kudu/127.4.83.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
I20260619 20:32:36.111464 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:0
--local_ip_for_outbound_sockets=127.4.83.66
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:40261
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:35633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:36.192025 14855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:36.192248 14855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:36.192271 14855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:36.193483 14855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:36.193535 14855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:36.193562 14855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:32:36.195072 14855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:35633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:40261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:36.195319 14855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:36.195533 14855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:36.200654 14861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:36.200639 14863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:36.200798 14855 server_base.cc:1034] running on GCE node
W20260619 20:32:36.200913 14860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:36.201123 14855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:36.201364 14855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:36.202531 14855 hybrid_clock.cc:648] HybridClock initialized: now 1781901156202517 us; error 36 us; skew 500 ppm
Jun 19 20:32:36 dist-test-slave-4m58 krb5kdc[14614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901156, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:36.205022 14855 init.cc:377] Logged in from keytab as kudu/127.4.83.66@KRBTEST.COM (short username kudu)
I20260619 20:32:36.205447 14855 webserver.cc:460] Webserver started at http://127.4.83.66:33145/ using document root <none> and password file <none>
I20260619 20:32:36.205632 14855 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:36.205672 14855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:36.205768 14855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:36.206756 14855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/data/instance:
uuid: "390e91893e574287aa4a2214a6bb24d0"
format_stamp: "Formatted at 2026-06-19 20:32:36 on dist-test-slave-4m58"
server_key: "23e19e61e76f10aff2ebf449e92478cc"
server_key_iv: "6c44f6b6921872fa4a5f00e510988f2b"
server_key_version: "encryptionkey@0"
I20260619 20:32:36.207047 14855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance:
uuid: "390e91893e574287aa4a2214a6bb24d0"
format_stamp: "Formatted at 2026-06-19 20:32:36 on dist-test-slave-4m58"
server_key: "23e19e61e76f10aff2ebf449e92478cc"
server_key_iv: "6c44f6b6921872fa4a5f00e510988f2b"
server_key_version: "encryptionkey@0"
I20260619 20:32:36.208381 14855 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:32:36.209097 14870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:36.209311 14855 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:32:36.209390 14855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "390e91893e574287aa4a2214a6bb24d0"
format_stamp: "Formatted at 2026-06-19 20:32:36 on dist-test-slave-4m58"
server_key: "23e19e61e76f10aff2ebf449e92478cc"
server_key_iv: "6c44f6b6921872fa4a5f00e510988f2b"
server_key_version: "encryptionkey@0"
I20260619 20:32:36.209446 14855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:36.216584 14855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:36.218636 14855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:36.218750 14855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:36.218930 14855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:36.219200 14855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:36.219241 14855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:36.219261 14855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:36.219272 14855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:36.224771 14855 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:46173
I20260619 20:32:36.224841 14983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:46173 every 8 connection(s)
I20260619 20:32:36.225113 14855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
I20260619 20:32:36.226336 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 14855
I20260619 20:32:36.226434 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance
I20260619 20:32:36.226603 4429 external_mini_cluster.cc:1428] Setting key 09cbb44bcd453a85d8c1de63c30e52e6
Jun 19 20:32:36 dist-test-slave-4m58 krb5kdc[14614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901156, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.83.67@KRBTEST.COM; defaulting to no policy
I20260619 20:32:36.235657 14716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:36.225814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:34513 (local address 127.4.83.126:40261)
0619 20:32:36.225876 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:36.225878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:36.226070 (+ 192us) server_negotiation.cc:407] Connection header received
0619 20:32:36.226617 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:36.226619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:36.226631 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:36.226704 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:36.227677 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.228239 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.229216 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.229470 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.229536 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:36.232684 (+ 3148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:36.232686 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:36.232686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:36.232694 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:36.234065 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:36.234631 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:36.234632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:36.234632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:36.234709 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:36.234993 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:36.234995 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:36.234995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:36.235121 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:36.235178 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:36.235455 (+ 277us) server_negotiation.cc:299] Negotiation successful
0619 20:32:36.235563 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 20:32:36.235819 14986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:36.225768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:40261 (local address 127.4.83.66:34513)
0619 20:32:36.225952 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:36.225991 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:36.226515 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:36.226750 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:36.226754 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:36.227389 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:36.227588 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:36.227594 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.228331 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.228332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:36.229133 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:36.229135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.229583 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.229585 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:36.229711 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:36.230255 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:36.230264 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:36.232507 (+ 2243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:36.234161 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:36.234171 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:36.234174 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:36.234519 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:36.234780 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:36.234781 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:36.234781 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:36.234928 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:36.235178 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:36.235185 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:36.235334 (+ 149us) client_negotiation.cc:769] Sending connection context
0619 20:32:36.235426 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 20:32:36.235597 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":87,"threads_started":1}
I20260619 20:32:36.236310 14984 heartbeater.cc:347] Connected to a master server at 127.4.83.126:40261
I20260619 20:32:36.236402 14984 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:36.236590 14984 heartbeater.cc:511] Master 127.4.83.126:40261 requested a full tablet report, sending...
I20260619 20:32:36.236917 14659 ts_manager.cc:194] Registered new tserver with Master: 390e91893e574287aa4a2214a6bb24d0 (127.4.83.66:46173)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.67@KRBTEST.COM" created.
I20260619 20:32:36.237322 14659 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.66@KRBTEST.COM'} at 127.4.83.66:34513
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
I20260619 20:32:36.251819 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:0
--local_ip_for_outbound_sockets=127.4.83.67
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:40261
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:35633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:32:36.334199 14989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:32:36.334431 14989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:32:36.334463 14989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:32:36.335603 14989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:32:36.335646 14989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:32:36.335682 14989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:32:36.337155 14989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:35633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:40261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:32:36.337394 14989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:32:36.337631 14989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:32:36.342468 14995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:36.342533 14989 server_base.cc:1034] running on GCE node
W20260619 20:32:36.342480 14997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:32:36.342485 14994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:32:36.343003 14989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:32:36.343262 14989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:32:36.344456 14989 hybrid_clock.cc:648] HybridClock initialized: now 1781901156344441 us; error 32 us; skew 500 ppm
Jun 19 20:32:36 dist-test-slave-4m58 krb5kdc[14614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901156, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:32:36.347024 14989 init.cc:377] Logged in from keytab as kudu/127.4.83.67@KRBTEST.COM (short username kudu)
I20260619 20:32:36.347502 14989 webserver.cc:460] Webserver started at http://127.4.83.67:46275/ using document root <none> and password file <none>
I20260619 20:32:36.347711 14989 fs_manager.cc:359] Metadata directory not provided
I20260619 20:32:36.347760 14989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:32:36.347911 14989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:32:36.348959 14989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/data/instance:
uuid: "c5beb9cecbb94cb0aad01ad5042b124f"
format_stamp: "Formatted at 2026-06-19 20:32:36 on dist-test-slave-4m58"
server_key: "75b4250bbf21cdffedf3b67209a04628"
server_key_iv: "d40ab9f4202fca506297495a57d01fdf"
server_key_version: "encryptionkey@0"
I20260619 20:32:36.349283 14989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance:
uuid: "c5beb9cecbb94cb0aad01ad5042b124f"
format_stamp: "Formatted at 2026-06-19 20:32:36 on dist-test-slave-4m58"
server_key: "75b4250bbf21cdffedf3b67209a04628"
server_key_iv: "d40ab9f4202fca506297495a57d01fdf"
server_key_version: "encryptionkey@0"
I20260619 20:32:36.350620 14989 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 20:32:36.351393 15004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:36.351562 14989 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:32:36.351652 14989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "c5beb9cecbb94cb0aad01ad5042b124f"
format_stamp: "Formatted at 2026-06-19 20:32:36 on dist-test-slave-4m58"
server_key: "75b4250bbf21cdffedf3b67209a04628"
server_key_iv: "d40ab9f4202fca506297495a57d01fdf"
server_key_version: "encryptionkey@0"
I20260619 20:32:36.351723 14989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:32:36.358541 14989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:32:36.360690 14989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:32:36.360834 14989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:32:36.361052 14989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:32:36.361359 14989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:32:36.361393 14989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:36.361423 14989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:32:36.361446 14989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:32:36.367072 14989 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:41093
I20260619 20:32:36.367126 15117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:41093 every 8 connection(s)
I20260619 20:32:36.367439 14989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
Jun 19 20:32:36 dist-test-slave-4m58 krb5kdc[14614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901156, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:32:36.376626 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 14989
I20260619 20:32:36.376677 14716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:36.368123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:60743 (local address 127.4.83.126:40261)
0619 20:32:36.368178 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:36.368180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:36.368367 (+ 187us) server_negotiation.cc:407] Connection header received
0619 20:32:36.368832 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:36.368833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:36.368842 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:36.368898 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:36.369832 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.370386 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.371355 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.371544 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.371616 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:36.373938 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:36.373940 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:36.373940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:36.373947 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:36.375109 (+ 1162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:36.375546 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:36.375547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:36.375547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:36.375607 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:36.375939 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:36.375940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:36.375941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:36.376074 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:36.376136 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:36.376488 (+ 352us) server_negotiation.cc:299] Negotiation successful
0619 20:32:36.376573 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 20:32:36.376730 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance
I20260619 20:32:36.376731 15120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:36.368087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:40261 (local address 127.4.83.67:60743)
0619 20:32:36.368244 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:36.368286 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:36.368758 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:36.368960 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:36.368963 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:36.369532 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:32:36.369730 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:36.369735 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.370466 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.370466 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:36.371258 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:36.371260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.371656 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.371657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:36.371785 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:36.372400 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:32:36.372409 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:32:36.373846 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:32:36.375205 (+ 1359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:36.375212 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:36.375243 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:36.375453 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:36.375693 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:32:36.375694 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:32:36.375694 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:32:36.375823 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:32:36.376157 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:32:36.376163 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:32:36.376327 (+ 164us) client_negotiation.cc:769] Sending connection context
0619 20:32:36.376417 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 20:32:36.376546 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":72,"threads_started":1}
I20260619 20:32:36.376911 4429 external_mini_cluster.cc:1428] Setting key 5f9e0f21950be7d5c7d99c58238a6c02
I20260619 20:32:36.377422 15118 heartbeater.cc:347] Connected to a master server at 127.4.83.126:40261
I20260619 20:32:36.377525 15118 heartbeater.cc:464] Registering TS with master...
I20260619 20:32:36.377707 15118 heartbeater.cc:511] Master 127.4.83.126:40261 requested a full tablet report, sending...
I20260619 20:32:36.378060 14658 ts_manager.cc:194] Registered new tserver with Master: c5beb9cecbb94cb0aad01ad5042b124f (127.4.83.67:41093)
I20260619 20:32:36.378450 14658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.67@KRBTEST.COM'} at 127.4.83.67:60743
I20260619 20:32:36.388844 4429 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:32:36.396315 14716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:36.390202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54102 (local address 127.4.83.126:40261)
0619 20:32:36.390252 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:36.390254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:36.390346 (+ 92us) server_negotiation.cc:407] Connection header received
0619 20:32:36.390407 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:36.390408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:36.390417 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:36.390468 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:32:36.391043 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.391570 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.392450 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.392657 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.392719 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:36.393996 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:32:36.393997 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:32:36.393998 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:32:36.394004 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:32:36.395265 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:36.395584 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:36.395585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:36.395585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:36.395633 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:32:36.395869 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:32:36.395870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:32:36.395870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:32:36.396025 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:32:36.396103 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:36.396184 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 20:32:36.396247 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 20:32:36.397922 14658 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54102:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:32:36.399422 14658 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:32:36.405761 15129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:36.401839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:46173 (local address 127.0.0.1:51980)
0619 20:32:36.402072 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:36.402082 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:36.402134 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:36.402414 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:36.402415 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:36.402416 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:36.402568 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:36.402571 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.403786 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.403787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:36.404617 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:36.404619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.405527 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.405528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:36.405588 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:36.405592 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:32:36.405621 (+ 29us) client_negotiation.cc:240] Negotiation successful
0619 20:32:36.405663 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":1,"thread_start_us":64,"threads_started":1}
I20260619 20:32:36.405803 15131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:36.401723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57868 (local address 127.4.83.67:41093)
0619 20:32:36.401875 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:36.401880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:36.402164 (+ 284us) server_negotiation.cc:407] Connection header received
0619 20:32:36.402208 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:36.402210 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:36.402224 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:36.402299 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:36.402615 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.403359 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.404807 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.405474 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.405538 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:36.405592 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:36.405689 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 20:32:36.405731 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
I20260619 20:32:36.405831 15132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:36.401754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60850 (local address 127.4.83.65:34981)
0619 20:32:36.402014 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:36.402018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:36.402166 (+ 148us) server_negotiation.cc:407] Connection header received
0619 20:32:36.402208 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:36.402210 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:36.402224 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:36.402298 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:36.402481 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.403602 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.404851 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.405321 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.405383 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:36.405444 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:36.405715 (+ 271us) server_negotiation.cc:299] Negotiation successful
0619 20:32:36.405757 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":85,"threads_started":1}
I20260619 20:32:36.405926 15130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:36.401623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:34981 (local address 127.0.0.1:60850)
0619 20:32:36.401965 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:36.402003 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:36.402080 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:36.402314 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:36.402316 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:36.402318 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:36.402395 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:36.402398 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.403697 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.403697 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:36.404794 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:36.404796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.405394 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.405395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:36.405481 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:36.405495 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 20:32:36.405560 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 20:32:36.405615 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":7,"thread_start_us":49,"threads_started":1}
I20260619 20:32:36.406002 15133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:36.401629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:41093 (local address 127.0.0.1:57868)
0619 20:32:36.402024 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:36.402032 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:36.402080 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:36.402375 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:36.402376 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:36.402378 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:36.402541 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:36.402543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.403450 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.403451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:36.404732 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:36.404735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.405545 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.405546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:36.405616 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:36.405619 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:32:36.405704 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 20:32:36.405740 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":84,"threads_started":1}
I20260619 20:32:36.406283 15134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:36.401998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51980 (local address 127.4.83.66:46173)
0619 20:32:36.402246 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:36.402249 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:36.402256 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:32:36.402292 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:36.402294 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:36.402309 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:36.402414 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:36.402650 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.403710 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.404724 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.405425 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.405806 (+ 381us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:36.406137 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:36.406163 (+ 26us) server_negotiation.cc:299] Negotiation successful
0619 20:32:36.406206 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":82,"threads_started":1}
I20260619 20:32:36.406703 15052 tablet_service.cc:1467] Processing CreateTablet for tablet 30c5d762e30b4152b72b62668e2ac57e (DEFAULT_TABLE table=test-table [id=f0e64cf70f8a48f69ccded831d896ed3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:32:36.406826 14784 tablet_service.cc:1467] Processing CreateTablet for tablet 30c5d762e30b4152b72b62668e2ac57e (DEFAULT_TABLE table=test-table [id=f0e64cf70f8a48f69ccded831d896ed3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:32:36.406920 14918 tablet_service.cc:1467] Processing CreateTablet for tablet 30c5d762e30b4152b72b62668e2ac57e (DEFAULT_TABLE table=test-table [id=f0e64cf70f8a48f69ccded831d896ed3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:32:36.407073 15052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30c5d762e30b4152b72b62668e2ac57e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:36.407155 14784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30c5d762e30b4152b72b62668e2ac57e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:36.407189 14918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30c5d762e30b4152b72b62668e2ac57e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:32:36.409734 15135 tablet_bootstrap.cc:492] T 30c5d762e30b4152b72b62668e2ac57e P 068d492b6b88455fbe366c0e277e2bde: Bootstrap starting.
I20260619 20:32:36.410454 15135 tablet_bootstrap.cc:654] T 30c5d762e30b4152b72b62668e2ac57e P 068d492b6b88455fbe366c0e277e2bde: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:36.410482 15137 tablet_bootstrap.cc:492] T 30c5d762e30b4152b72b62668e2ac57e P 390e91893e574287aa4a2214a6bb24d0: Bootstrap starting.
I20260619 20:32:36.410897 15135 log.cc:824] T 30c5d762e30b4152b72b62668e2ac57e P 068d492b6b88455fbe366c0e277e2bde: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:36.410969 15137 tablet_bootstrap.cc:654] T 30c5d762e30b4152b72b62668e2ac57e P 390e91893e574287aa4a2214a6bb24d0: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:36.411470 15137 log.cc:824] T 30c5d762e30b4152b72b62668e2ac57e P 390e91893e574287aa4a2214a6bb24d0: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:36.411553 15135 tablet_bootstrap.cc:492] T 30c5d762e30b4152b72b62668e2ac57e P 068d492b6b88455fbe366c0e277e2bde: No bootstrap required, opened a new log
I20260619 20:32:36.411633 15135 ts_tablet_manager.cc:1397] T 30c5d762e30b4152b72b62668e2ac57e P 068d492b6b88455fbe366c0e277e2bde: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:32:36.411788 15136 tablet_bootstrap.cc:492] T 30c5d762e30b4152b72b62668e2ac57e P c5beb9cecbb94cb0aad01ad5042b124f: Bootstrap starting.
I20260619 20:32:36.412391 15136 tablet_bootstrap.cc:654] T 30c5d762e30b4152b72b62668e2ac57e P c5beb9cecbb94cb0aad01ad5042b124f: Neither blocks nor log segments found. Creating new log.
I20260619 20:32:36.412833 15136 log.cc:824] T 30c5d762e30b4152b72b62668e2ac57e P c5beb9cecbb94cb0aad01ad5042b124f: Log is configured to *not* fsync() on all Append() calls
I20260619 20:32:36.413321 15137 tablet_bootstrap.cc:492] T 30c5d762e30b4152b72b62668e2ac57e P 390e91893e574287aa4a2214a6bb24d0: No bootstrap required, opened a new log
I20260619 20:32:36.413393 15137 ts_tablet_manager.cc:1397] T 30c5d762e30b4152b72b62668e2ac57e P 390e91893e574287aa4a2214a6bb24d0: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 20:32:36.413381 15135 raft_consensus.cc:348] T 30c5d762e30b4152b72b62668e2ac57e P 068d492b6b88455fbe366c0e277e2bde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068d492b6b88455fbe366c0e277e2bde" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34981 } } peers { permanent_uuid: "c5beb9cecbb94cb0aad01ad5042b124f" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41093 } } peers { permanent_uuid: "390e91893e574287aa4a2214a6bb24d0" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 46173 } }
I20260619 20:32:36.413522 15135 raft_consensus.cc:374] T 30c5d762e30b4152b72b62668e2ac57e P 068d492b6b88455fbe366c0e277e2bde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:36.413553 15135 raft_consensus.cc:729] T 30c5d762e30b4152b72b62668e2ac57e P 068d492b6b88455fbe366c0e277e2bde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068d492b6b88455fbe366c0e277e2bde, State: Initialized, Role: FOLLOWER
I20260619 20:32:36.413712 15136 tablet_bootstrap.cc:492] T 30c5d762e30b4152b72b62668e2ac57e P c5beb9cecbb94cb0aad01ad5042b124f: No bootstrap required, opened a new log
I20260619 20:32:36.413699 15135 consensus_queue.cc:260] T 30c5d762e30b4152b72b62668e2ac57e P 068d492b6b88455fbe366c0e277e2bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068d492b6b88455fbe366c0e277e2bde" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34981 } } peers { permanent_uuid: "c5beb9cecbb94cb0aad01ad5042b124f" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41093 } } peers { permanent_uuid: "390e91893e574287aa4a2214a6bb24d0" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 46173 } }
I20260619 20:32:36.413779 15136 ts_tablet_manager.cc:1397] T 30c5d762e30b4152b72b62668e2ac57e P c5beb9cecbb94cb0aad01ad5042b124f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:32:36.413957 15135 ts_tablet_manager.cc:1428] T 30c5d762e30b4152b72b62668e2ac57e P 068d492b6b88455fbe366c0e277e2bde: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 20:32:36.414084 14850 heartbeater.cc:503] Master 127.4.83.126:40261 was elected leader, sending a full tablet report...
I20260619 20:32:36.415318 15137 raft_consensus.cc:348] T 30c5d762e30b4152b72b62668e2ac57e P 390e91893e574287aa4a2214a6bb24d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068d492b6b88455fbe366c0e277e2bde" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34981 } } peers { permanent_uuid: "c5beb9cecbb94cb0aad01ad5042b124f" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41093 } } peers { permanent_uuid: "390e91893e574287aa4a2214a6bb24d0" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 46173 } }
I20260619 20:32:36.415488 15137 raft_consensus.cc:374] T 30c5d762e30b4152b72b62668e2ac57e P 390e91893e574287aa4a2214a6bb24d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:36.415519 15137 raft_consensus.cc:729] T 30c5d762e30b4152b72b62668e2ac57e P 390e91893e574287aa4a2214a6bb24d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 390e91893e574287aa4a2214a6bb24d0, State: Initialized, Role: FOLLOWER
I20260619 20:32:36.415627 15137 consensus_queue.cc:260] T 30c5d762e30b4152b72b62668e2ac57e P 390e91893e574287aa4a2214a6bb24d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068d492b6b88455fbe366c0e277e2bde" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34981 } } peers { permanent_uuid: "c5beb9cecbb94cb0aad01ad5042b124f" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41093 } } peers { permanent_uuid: "390e91893e574287aa4a2214a6bb24d0" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 46173 } }
I20260619 20:32:36.415644 15136 raft_consensus.cc:348] T 30c5d762e30b4152b72b62668e2ac57e P c5beb9cecbb94cb0aad01ad5042b124f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068d492b6b88455fbe366c0e277e2bde" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34981 } } peers { permanent_uuid: "c5beb9cecbb94cb0aad01ad5042b124f" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41093 } } peers { permanent_uuid: "390e91893e574287aa4a2214a6bb24d0" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 46173 } }
I20260619 20:32:36.415779 15136 raft_consensus.cc:374] T 30c5d762e30b4152b72b62668e2ac57e P c5beb9cecbb94cb0aad01ad5042b124f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:32:36.415818 15136 raft_consensus.cc:729] T 30c5d762e30b4152b72b62668e2ac57e P c5beb9cecbb94cb0aad01ad5042b124f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5beb9cecbb94cb0aad01ad5042b124f, State: Initialized, Role: FOLLOWER
I20260619 20:32:36.415944 14984 heartbeater.cc:503] Master 127.4.83.126:40261 was elected leader, sending a full tablet report...
I20260619 20:32:36.415969 15136 consensus_queue.cc:260] T 30c5d762e30b4152b72b62668e2ac57e P c5beb9cecbb94cb0aad01ad5042b124f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068d492b6b88455fbe366c0e277e2bde" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34981 } } peers { permanent_uuid: "c5beb9cecbb94cb0aad01ad5042b124f" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41093 } } peers { permanent_uuid: "390e91893e574287aa4a2214a6bb24d0" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 46173 } }
I20260619 20:32:36.415939 15137 ts_tablet_manager.cc:1428] T 30c5d762e30b4152b72b62668e2ac57e P 390e91893e574287aa4a2214a6bb24d0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:32:36.416220 15136 ts_tablet_manager.cc:1428] T 30c5d762e30b4152b72b62668e2ac57e P c5beb9cecbb94cb0aad01ad5042b124f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 20:32:36.416371 15118 heartbeater.cc:503] Master 127.4.83.126:40261 was elected leader, sending a full tablet report...
W20260619 20:32:36.475718 14985 tablet.cc:2369] T 30c5d762e30b4152b72b62668e2ac57e P 390e91893e574287aa4a2214a6bb24d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:32:36.495328 15141 raft_consensus.cc:482] T 30c5d762e30b4152b72b62668e2ac57e P 068d492b6b88455fbe366c0e277e2bde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:32:36.495465 15141 raft_consensus.cc:504] T 30c5d762e30b4152b72b62668e2ac57e P 068d492b6b88455fbe366c0e277e2bde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068d492b6b88455fbe366c0e277e2bde" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34981 } } peers { permanent_uuid: "c5beb9cecbb94cb0aad01ad5042b124f" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41093 } } peers { permanent_uuid: "390e91893e574287aa4a2214a6bb24d0" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 46173 } }
I20260619 20:32:36.495771 15141 leader_election.cc:288] T 30c5d762e30b4152b72b62668e2ac57e P 068d492b6b88455fbe366c0e277e2bde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c5beb9cecbb94cb0aad01ad5042b124f (127.4.83.67:41093), 390e91893e574287aa4a2214a6bb24d0 (127.4.83.66:46173)
I20260619 20:32:36.499307 15134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:36.496177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:51433 (local address 127.4.83.66:46173)
0619 20:32:36.496218 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:36.496220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:36.496348 (+ 128us) server_negotiation.cc:407] Connection header received
0619 20:32:36.496386 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:36.496387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:36.496397 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:36.496437 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:36.496559 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.497245 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.498309 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.498922 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.498997 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:36.499009 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:36.499187 (+ 178us) server_negotiation.cc:299] Negotiation successful
0619 20:32:36.499237 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:32:36.499394 15131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:36.496106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:37973 (local address 127.4.83.67:41093)
0619 20:32:36.496156 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:36.496158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:36.496169 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:32:36.496199 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:36.496199 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:36.496210 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:36.496289 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:32:36.496461 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.497199 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.498323 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.498928 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.498998 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:36.499009 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:36.499297 (+ 288us) server_negotiation.cc:299] Negotiation successful
0619 20:32:36.499338 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 20:32:36.499382 14852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:36.495990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:41093 (local address 127.4.83.65:37973)
0619 20:32:36.496051 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:36.496062 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:36.496127 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:36.496299 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:36.496300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:36.496302 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:36.496392 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:36.496394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.497340 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.497341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:36.498207 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:36.498210 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.499030 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.499030 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:36.499106 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:36.499110 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:32:36.499167 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 20:32:36.499248 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":27}
I20260619 20:32:36.499399 15144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:36.496092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:46173 (local address 127.4.83.65:51433)
0619 20:32:36.496276 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:36.496285 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:32:36.496336 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:32:36.496451 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:32:36.496452 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:32:36.496453 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:32:36.496506 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:36.496507 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.497311 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.497312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:36.498205 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:32:36.498209 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.499056 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.499057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:32:36.499117 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:36.499120 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:32:36.499165 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 20:32:36.499216 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":4,"thread_start_us":130,"threads_started":1}
I20260619 20:32:36.499786 14938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30c5d762e30b4152b72b62668e2ac57e" candidate_uuid: "068d492b6b88455fbe366c0e277e2bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "390e91893e574287aa4a2214a6bb24d0" is_pre_election: true
I20260619 20:32:36.499797 15072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30c5d762e30b4152b72b62668e2ac57e" candidate_uuid: "068d492b6b88455fbe366c0e277e2bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5beb9cecbb94cb0aad01ad5042b124f" is_pre_election: true
I20260619 20:32:36.499984 15072 raft_consensus.cc:2454] T 30c5d762e30b4152b72b62668e2ac57e P c5beb9cecbb94cb0aad01ad5042b124f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 068d492b6b88455fbe366c0e277e2bde in term 0.
I20260619 20:32:36.499984 14938 raft_consensus.cc:2454] T 30c5d762e30b4152b72b62668e2ac57e P 390e91893e574287aa4a2214a6bb24d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 068d492b6b88455fbe366c0e277e2bde in term 0.
I20260619 20:32:36.500195 14740 leader_election.cc:302] T 30c5d762e30b4152b72b62668e2ac57e P 068d492b6b88455fbe366c0e277e2bde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068d492b6b88455fbe366c0e277e2bde, 390e91893e574287aa4a2214a6bb24d0; no voters:
I20260619 20:32:36.500315 15141 raft_consensus.cc:2781] T 30c5d762e30b4152b72b62668e2ac57e P 068d492b6b88455fbe366c0e277e2bde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:32:36.500373 15141 raft_consensus.cc:482] T 30c5d762e30b4152b72b62668e2ac57e P 068d492b6b88455fbe366c0e277e2bde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:32:36.500396 15141 raft_consensus.cc:3037] T 30c5d762e30b4152b72b62668e2ac57e P 068d492b6b88455fbe366c0e277e2bde [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:36.501130 15141 raft_consensus.cc:504] T 30c5d762e30b4152b72b62668e2ac57e P 068d492b6b88455fbe366c0e277e2bde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068d492b6b88455fbe366c0e277e2bde" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34981 } } peers { permanent_uuid: "c5beb9cecbb94cb0aad01ad5042b124f" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41093 } } peers { permanent_uuid: "390e91893e574287aa4a2214a6bb24d0" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 46173 } }
I20260619 20:32:36.501245 15141 leader_election.cc:288] T 30c5d762e30b4152b72b62668e2ac57e P 068d492b6b88455fbe366c0e277e2bde [CANDIDATE]: Term 1 election: Requested vote from peers c5beb9cecbb94cb0aad01ad5042b124f (127.4.83.67:41093), 390e91893e574287aa4a2214a6bb24d0 (127.4.83.66:46173)
I20260619 20:32:36.501412 15072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30c5d762e30b4152b72b62668e2ac57e" candidate_uuid: "068d492b6b88455fbe366c0e277e2bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5beb9cecbb94cb0aad01ad5042b124f"
I20260619 20:32:36.501461 14938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30c5d762e30b4152b72b62668e2ac57e" candidate_uuid: "068d492b6b88455fbe366c0e277e2bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "390e91893e574287aa4a2214a6bb24d0"
I20260619 20:32:36.501490 15072 raft_consensus.cc:3037] T 30c5d762e30b4152b72b62668e2ac57e P c5beb9cecbb94cb0aad01ad5042b124f [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:36.501536 14938 raft_consensus.cc:3037] T 30c5d762e30b4152b72b62668e2ac57e P 390e91893e574287aa4a2214a6bb24d0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:32:36.502494 14938 raft_consensus.cc:2454] T 30c5d762e30b4152b72b62668e2ac57e P 390e91893e574287aa4a2214a6bb24d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068d492b6b88455fbe366c0e277e2bde in term 1.
I20260619 20:32:36.502494 15072 raft_consensus.cc:2454] T 30c5d762e30b4152b72b62668e2ac57e P c5beb9cecbb94cb0aad01ad5042b124f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068d492b6b88455fbe366c0e277e2bde in term 1.
I20260619 20:32:36.502688 14740 leader_election.cc:302] T 30c5d762e30b4152b72b62668e2ac57e P 068d492b6b88455fbe366c0e277e2bde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068d492b6b88455fbe366c0e277e2bde, 390e91893e574287aa4a2214a6bb24d0; no voters:
I20260619 20:32:36.502779 15141 raft_consensus.cc:2781] T 30c5d762e30b4152b72b62668e2ac57e P 068d492b6b88455fbe366c0e277e2bde [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:32:36.502929 15141 raft_consensus.cc:686] T 30c5d762e30b4152b72b62668e2ac57e P 068d492b6b88455fbe366c0e277e2bde [term 1 LEADER]: Becoming Leader. State: Replica: 068d492b6b88455fbe366c0e277e2bde, State: Running, Role: LEADER
I20260619 20:32:36.503022 15141 consensus_queue.cc:237] T 30c5d762e30b4152b72b62668e2ac57e P 068d492b6b88455fbe366c0e277e2bde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068d492b6b88455fbe366c0e277e2bde" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34981 } } peers { permanent_uuid: "c5beb9cecbb94cb0aad01ad5042b124f" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41093 } } peers { permanent_uuid: "390e91893e574287aa4a2214a6bb24d0" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 46173 } }
I20260619 20:32:36.503979 14659 catalog_manager.cc:5495] T 30c5d762e30b4152b72b62668e2ac57e P 068d492b6b88455fbe366c0e277e2bde reported cstate change: term changed from 0 to 1, leader changed from <none> to 068d492b6b88455fbe366c0e277e2bde (127.4.83.65). New cstate: current_term: 1 leader_uuid: "068d492b6b88455fbe366c0e277e2bde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068d492b6b88455fbe366c0e277e2bde" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5beb9cecbb94cb0aad01ad5042b124f" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "390e91893e574287aa4a2214a6bb24d0" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 46173 } health_report { overall_health: UNKNOWN } } }
I20260619 20:32:36.525641 15132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:36.523177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60858 (local address 127.4.83.65:34981)
0619 20:32:36.523247 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 20:32:36.523249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:32:36.523257 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:32:36.523300 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:32:36.523301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:32:36.523322 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:32:36.523387 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:32:36.523599 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.524126 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.524885 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:32:36.525073 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:32:36.525133 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:32:36.525302 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:32:36.525412 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:32:36.525485 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 20:32:36.525547 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 20:32:36.525566 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 20:32:36.527426 14938 raft_consensus.cc:1261] T 30c5d762e30b4152b72b62668e2ac57e P 390e91893e574287aa4a2214a6bb24d0 [term 1 FOLLOWER]: Refusing update from remote peer 068d492b6b88455fbe366c0e277e2bde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:32:36.527454 15072 raft_consensus.cc:1261] T 30c5d762e30b4152b72b62668e2ac57e P c5beb9cecbb94cb0aad01ad5042b124f [term 1 FOLLOWER]: Refusing update from remote peer 068d492b6b88455fbe366c0e277e2bde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:32:36.527788 15141 consensus_queue.cc:1035] T 30c5d762e30b4152b72b62668e2ac57e P 068d492b6b88455fbe366c0e277e2bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5beb9cecbb94cb0aad01ad5042b124f" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:32:36.527964 15145 consensus_queue.cc:1035] T 30c5d762e30b4152b72b62668e2ac57e P 068d492b6b88455fbe366c0e277e2bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "390e91893e574287aa4a2214a6bb24d0" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 46173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:32:36.531314 15150 mvcc.cc:205] Tried to move back new op lower bound from 7298667137133780992 to 7298667137037008896. Current Snapshot: MvccSnapshot[applied={T|T < 7298667137133780992 or (T in {7298667137133780992})}]
I20260619 20:32:36.533526 14657 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:32:36.533628 14657 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:32:36.534468 14657 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 402047a7c26f4f009325fda6251bc65f: Sending DeleteTablet for 3 replicas of tablet 30c5d762e30b4152b72b62668e2ac57e
I20260619 20:32:36.534849 14784 tablet_service.cc:1514] Processing DeleteTablet for tablet 30c5d762e30b4152b72b62668e2ac57e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:32:36 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:60850
I20260619 20:32:36.534898 15052 tablet_service.cc:1514] Processing DeleteTablet for tablet 30c5d762e30b4152b72b62668e2ac57e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:32:36 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:57868
I20260619 20:32:36.535105 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 14721
I20260619 20:32:36.535131 14918 tablet_service.cc:1514] Processing DeleteTablet for tablet 30c5d762e30b4152b72b62668e2ac57e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:32:36 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:51980
I20260619 20:32:36.535851 15160 tablet_replica.cc:330] T 30c5d762e30b4152b72b62668e2ac57e P 390e91893e574287aa4a2214a6bb24d0: stopping tablet replica
I20260619 20:32:36.535979 15160 raft_consensus.cc:2229] T 30c5d762e30b4152b72b62668e2ac57e P 390e91893e574287aa4a2214a6bb24d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:32:36.536090 15160 raft_consensus.cc:2258] T 30c5d762e30b4152b72b62668e2ac57e P 390e91893e574287aa4a2214a6bb24d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:32:36.536267 15159 tablet_replica.cc:330] T 30c5d762e30b4152b72b62668e2ac57e P c5beb9cecbb94cb0aad01ad5042b124f: stopping tablet replica
I20260619 20:32:36.536373 15159 raft_consensus.cc:2229] T 30c5d762e30b4152b72b62668e2ac57e P c5beb9cecbb94cb0aad01ad5042b124f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:32:36.536471 15159 raft_consensus.cc:2258] T 30c5d762e30b4152b72b62668e2ac57e P c5beb9cecbb94cb0aad01ad5042b124f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:32:36.536929 15159 ts_tablet_manager.cc:1905] T 30c5d762e30b4152b72b62668e2ac57e P c5beb9cecbb94cb0aad01ad5042b124f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:32:36.538894 15159 ts_tablet_manager.cc:1918] T 30c5d762e30b4152b72b62668e2ac57e P c5beb9cecbb94cb0aad01ad5042b124f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:32:36.538980 15159 log.cc:1196] T 30c5d762e30b4152b72b62668e2ac57e P c5beb9cecbb94cb0aad01ad5042b124f: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901102012824-4429-0/minicluster-data/ts-2/wal/wals/30c5d762e30b4152b72b62668e2ac57e
I20260619 20:32:36.539242 15159 ts_tablet_manager.cc:1939] T 30c5d762e30b4152b72b62668e2ac57e P c5beb9cecbb94cb0aad01ad5042b124f: Deleting consensus metadata
I20260619 20:32:36.539774 14646 catalog_manager.cc:4841] TS c5beb9cecbb94cb0aad01ad5042b124f (127.4.83.67:41093): tablet 30c5d762e30b4152b72b62668e2ac57e (table test-table [id=f0e64cf70f8a48f69ccded831d896ed3]) successfully deleted
W20260619 20:32:36.540823 14644 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.83.65:34981 (error 108)
I20260619 20:32:36.540946 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 14855
I20260619 20:32:36.542071 15133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:36.541942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:34981 (local address 127.0.0.1:60872)
0619 20:32:36.541998 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:36.542036 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.65:34981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 20:32:36.542229 14644 catalog_manager.cc:4584] TS 068d492b6b88455fbe366c0e277e2bde (127.4.83.65:34981): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 30c5d762e30b4152b72b62668e2ac57e: Network error: Client connection negotiation failed: client connection to 127.4.83.65:34981: connect: Connection refused (error 111)
I20260619 20:32:36.545117 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 14989
I20260619 20:32:36.545148 15133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:32:36.545033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:46173 (local address 127.0.0.1:51984)
0619 20:32:36.545081 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 20:32:36.545111 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.66:46173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260619 20:32:36.549063 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 14627
2026-06-19T20:32:36Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1916 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:32:36 dist-test-slave-4m58 krb5kdc[15164](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:32:36 dist-test-slave-4m58 krb5kdc[15164](info): set up 2 sockets
Jun 19 20:32:36 dist-test-slave-4m58 krb5kdc[15164](info): commencing operation
krb5kdc: starting...
W20260619 20:32:37.529394 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.961s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:32:37 dist-test-slave-4m58 krb5kdc[15164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901157, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:32:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:32:37Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.4.83.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.4.83.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.4.83.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/rangeradmin_127.4.83.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.4.83.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/rangeradmin_127.4.83.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.4.83.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.4.83.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.4.83.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/rangerlookup_127.4.83.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.4.83.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/rangerlookup_127.4.83.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.83.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.4.83.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.83.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/HTTP_127.4.83.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.83.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/HTTP_127.4.83.84@KRBTEST.COM.keytab.
I20260619 20:32:37.692920 4429 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskPZCFKi/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/postgres -l logfile start
2026-06-19 20:32:40.277 UTC [15194] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 20:32:40.277 UTC [15194] LOG: listening on IPv4 address "127.4.83.84", port 36565
2026-06-19 20:32:40.280 UTC [15194] LOG: listening on Unix socket "/tmp/.s.PGSQL.36565"
2026-06-19 20:32:40.301 UTC [15197] LOG: database system was shut down at 2026-06-19 20:32:38 UTC
2026-06-19 20:32:40.305 UTC [15194] LOG: database system is ready to accept connections
I20260619 20:32:41.185531 4429 mini_postgres.cc:96] Postgres bound to 36565
2026-06-19 20:32:41.192 UTC [15205] FATAL: database "slave" does not exist
127.4.83.84:36565 - accepting connections
I20260619 20:32:41.192950 4429 mini_ranger.cc:161] Starting Ranger...
I20260619 20:32:41.207971 4429 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 20:32:41.747074 4429 mini_ranger.cc:87] Created ranger Postgres database
I20260619 20:32:41.747185 4429 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ranger-admin
2026-06-19 20:32:42,086 [I] DB FLAVOR :POSTGRES
2026-06-19 20:32:42,087 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 20:32:42,087 [I] Checking connection..
2026-06-19 20:32:42,087 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:36565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 20:32:42,390 [I] Checking connection passed.
2026-06-19 20:32:42,390 [I] --------- Verifying version history table ---------
2026-06-19 20:32:42,391 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:36565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 20:32:42,665 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 20:32:42,666 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:36565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 20:32:42,900 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 20:32:42,900 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 20:32:42,900 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:36565/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 20:32:43.162 UTC [15269] WARNING: there is no transaction in progress
2026-06-19 20:32:43,168 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 20:32:43,168 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:36565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 20:32:43,420 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 20:32:43,421 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 20:32:43,421 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:36565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 20:32:43,670 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:36565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 20:32:43,915 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:36565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-4m58.c.gcp-upstream.internal','N') ;"
2026-06-19 20:32:44,167 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:36565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 20:32:44,404 [I] Table x_portal_user does not exist in database ranger
2026-06-19 20:32:44,404 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:36565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 20:32:44,650 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 20:32:44,651 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:36565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 20:32:44,908 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:36565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 20:32:45,182 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 20:32:45,182 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:36565/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 20:32:46.263 UTC [15377] WARNING: there is no transaction in progress
2026-06-19 20:32:46.282 UTC [15377] WARNING: there is no transaction in progress
2026-06-19 20:32:46.304 UTC [15377] WARNING: there is no transaction in progress
2026-06-19 20:32:46.322 UTC [15377] WARNING: there is no transaction in progress
2026-06-19 20:32:46.342 UTC [15377] WARNING: there is no transaction in progress
2026-06-19 20:32:46.364 UTC [15377] WARNING: there is no transaction in progress
2026-06-19 20:32:46.428 UTC [15377] WARNING: there is no transaction in progress
2026-06-19 20:32:46.436 UTC [15377] WARNING: there is no transaction in progress
2026-06-19 20:32:46.453 UTC [15377] WARNING: there is no transaction in progress
2026-06-19 20:32:46.465 UTC [15377] WARNING: there is no transaction in progress
2026-06-19 20:32:46.476 UTC [15377] WARNING: there is no transaction in progress
2026-06-19 20:32:46.492 UTC [15377] WARNING: there is no transaction in progress
2026-06-19 20:32:46.503 UTC [15377] WARNING: there is no transaction in progress
2026-06-19 20:32:46.511 UTC [15377] WARNING: there is no transaction in progress
2026-06-19 20:32:46.520 UTC [15377] WARNING: there is no transaction in progress
2026-06-19 20:32:46.529 UTC [15377] WARNING: there is no transaction in progress
2026-06-19 20:32:47,375 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 20:32:47,376 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:36565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 20:32:47,661 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 20:32:47,661 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:36565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 20:32:47,959 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 20:32:47,960 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:36565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 20:32:48,269 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 20:32:48,270 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:36565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 20:32:48,581 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:36565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 20:32:48,836 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:36565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-4m58.c.gcp-upstream.internal';"
2026-06-19 20:32:49,059 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 20:32:49,060 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:36565/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 20:32:49,320 [I] DB_PATCHES have already been applied
I20260619 20:32:49.326934 4429 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ranger-admin:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskPZCFKi/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 20:32:49.327018 4429 mini_ranger.cc:193] Using host: 127.4.83.84
I20260619 20:32:49.330258 4429 mini_ranger.cc:227] Ranger admin URL: http://127.4.83.84:50763
Jun 19, 2026 8:32:49 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 8:32:49 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 8:32:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 8:32:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 8:32:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 8:32:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 8:32:50 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 20:32:50.352772 4429 mini_ranger.cc:160] Time spent starting Ranger: real 9.160s user 0.000s sys 0.009s
Jun 19, 2026 8:33:13 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 8:33:13 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 19, 2026 8:33:13 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 8:33:13 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 8:33:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 8:33:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 8:33:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 8:33:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 8:33:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 8:33:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 8:33:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 8:33:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 8:33:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 8:33:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 8:33:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 8:33:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 8:33:13 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 8:33:14 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 20:33:14.941195 4429 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.4.83.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.4.83.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.4.83.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/rangerkms_127.4.83.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.4.83.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/rangerkms_127.4.83.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.83.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.4.83.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.83.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/HTTP_127.4.83.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.83.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/HTTP_127.4.83.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 20:33:15 dist-test-slave-4m58 krb5kdc[15164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901195, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 20:33:15.024855 4429 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 20:33:15.034499 4429 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 20:33:16.174595 4429 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 20:33:16.174728 4429 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ranger-kms
I20260619 20:33:16.174744 4429 mini_ranger_kms.cc:227] Using postgres at 127.4.83.84:36565
2026-06-19 20:33:16,409 [I] DB FLAVOR :POSTGRES
2026-06-19 20:33:16,409 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 20:33:16,409 [I] Checking connection
2026-06-19 20:33:16,409 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:36565/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 20:33:16,696 [I] connection success
2026-06-19 20:33:16,697 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 20:33:16,697 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 20:33:16,697 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:36565/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 20:33:17,000 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 20:33:17,000 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 20:33:17,001 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 20:33:17,001 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:36565/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 20:33:17,303 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 20:33:17.785844 4429 mini_ranger_kms.cc:310] Created kms service
I20260619 20:33:18.030351 4429 mini_ranger_kms.cc:326] Created kudu user
I20260619 20:33:18.083351 4429 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 20:33:18 dist-test-slave-4m58 krb5kdc[15164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901195, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.4.83.84@KRBTEST.COM
I20260619 20:33:18.504770 4429 mini_ranger_kms.cc:384] Added ranger policy
I20260619 20:33:18.504885 4429 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ranger-kms:/tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 20:33:18.504904 4429 mini_ranger_kms.cc:240] Using host: 127.4.83.84
I20260619 20:33:18.508643 4429 mini_ranger_kms.cc:276] Ranger KMS PID: 15558
I20260619 20:33:18.508877 4429 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.4.83.84:38361
Jun 19, 2026 8:33:19 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 8:33:19 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 8:33:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 8:33:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 8:33:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 8:33:20 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 20:33:20.572849 4429 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.548s user 0.000s sys 0.216s
I20260619 20:33:20.572961 4429 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 20:33:20.573035 4429 mini_ranger_kms.cc:401] 127.4.83.84:38361/kms/v1/keys
Jun 19 20:33:20 dist-test-slave-4m58 krb5kdc[15164](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781901200, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.83.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 20:33:21 dist-test-slave-4m58 krb5kdc[15164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901200, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.83.84@KRBTEST.COM for HTTP/127.4.83.84@KRBTEST.COM
Jun 19 20:33:21 dist-test-slave-4m58 krb5kdc[15164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901200, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.83.84@KRBTEST.COM for HTTP/127.4.83.84@KRBTEST.COM
Jun 19 20:33:24 dist-test-slave-4m58 krb5kdc[15164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
I20260619 20:33:24.203426 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:35845
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:42615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:35845
--ranger_config_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.83.84:38361/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:33:24.297819 15608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:33:24.298080 15608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:33:24.298115 15608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:33:24.300954 15608 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:33:24.301050 15608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:33:24.301075 15608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:33:24.301092 15608 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:33:24.301110 15608 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:33:24.304050 15608 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:42615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/master-0/wal
--ranger_kms_url=127.4.83.84:38361/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:35845
--ranger_config_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:35845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:33:24.304388 15608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:33:24.304638 15608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:33:24.317227 15613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:33:24.317396 15608 server_base.cc:1034] running on GCE node
W20260619 20:33:24.317867 15616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:33:24.318225 15614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:33:24.318720 15608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:33:24.319085 15608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:33:24.320611 15608 hybrid_clock.cc:648] HybridClock initialized: now 1781901204320577 us; error 38 us; skew 500 ppm
Jun 19 20:33:24 dist-test-slave-4m58 krb5kdc[15164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901204, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:33:24.324549 15608 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:33:24.326225 15608 webserver.cc:460] Webserver started at http://127.4.83.126:44023/ using document root <none> and password file <none>
I20260619 20:33:24.327194 15608 fs_manager.cc:359] Metadata directory not provided
I20260619 20:33:24.327814 15608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:33:24.328054 15608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 20:33:24 dist-test-slave-4m58 krb5kdc[15164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901204, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for HTTP/127.4.83.84@KRBTEST.COM
I20260619 20:33:24.405401 15608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "e0b1de5761db4ecf86381d85992b7e39"
format_stamp: "Formatted at 2026-06-19 20:33:24 on dist-test-slave-4m58"
server_key: "2802010390200a56a7f69795c2f62193"
server_key_iv: "35babbf19a73d26e6c7b118fafb12e98"
server_key_version: "kuduclusterkey@0"
I20260619 20:33:24.405777 15608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "e0b1de5761db4ecf86381d85992b7e39"
format_stamp: "Formatted at 2026-06-19 20:33:24 on dist-test-slave-4m58"
server_key: "2802010390200a56a7f69795c2f62193"
server_key_iv: "35babbf19a73d26e6c7b118fafb12e98"
server_key_version: "kuduclusterkey@0"
I20260619 20:33:24.407152 15608 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 20:33:24.419221 15623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:33:24.419394 15608 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:33:24.419477 15608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "e0b1de5761db4ecf86381d85992b7e39"
format_stamp: "Formatted at 2026-06-19 20:33:24 on dist-test-slave-4m58"
server_key: "2802010390200a56a7f69795c2f62193"
server_key_iv: "35babbf19a73d26e6c7b118fafb12e98"
server_key_version: "kuduclusterkey@0"
I20260619 20:33:24.419574 15608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:33:24.477025 15608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:33:24.479759 15608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:33:24.479954 15608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:33:24.484696 15608 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:35845
I20260619 20:33:24.484724 15675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:35845 every 8 connection(s)
I20260619 20:33:24.485162 15608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:33:24.486269 15676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:33:24.489594 15676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0b1de5761db4ecf86381d85992b7e39: Bootstrap starting.
I20260619 20:33:24.490352 15676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0b1de5761db4ecf86381d85992b7e39: Neither blocks nor log segments found. Creating new log.
I20260619 20:33:24.490674 15676 log.cc:824] T 00000000000000000000000000000000 P e0b1de5761db4ecf86381d85992b7e39: Log is configured to *not* fsync() on all Append() calls
I20260619 20:33:24.491583 15676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0b1de5761db4ecf86381d85992b7e39: No bootstrap required, opened a new log
I20260619 20:33:24.492059 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 15608
I20260619 20:33:24.492158 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:33:24.492357 4429 external_mini_cluster.cc:1428] Setting key 02282b29ba0a207c8ddcbdbfe8dc0bb9
I20260619 20:33:24.494848 15676 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0b1de5761db4ecf86381d85992b7e39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0b1de5761db4ecf86381d85992b7e39" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35845 } }
I20260619 20:33:24.495110 15676 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0b1de5761db4ecf86381d85992b7e39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:33:24.495270 15676 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0b1de5761db4ecf86381d85992b7e39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0b1de5761db4ecf86381d85992b7e39, State: Initialized, Role: FOLLOWER
I20260619 20:33:24.495450 15676 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0b1de5761db4ecf86381d85992b7e39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0b1de5761db4ecf86381d85992b7e39" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35845 } }
I20260619 20:33:24.495535 15676 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0b1de5761db4ecf86381d85992b7e39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:33:24.495565 15676 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0b1de5761db4ecf86381d85992b7e39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:33:24.495627 15676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0b1de5761db4ecf86381d85992b7e39 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:33:24.496483 15676 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0b1de5761db4ecf86381d85992b7e39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0b1de5761db4ecf86381d85992b7e39" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35845 } }
I20260619 20:33:24.496603 15676 leader_election.cc:302] T 00000000000000000000000000000000 P e0b1de5761db4ecf86381d85992b7e39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0b1de5761db4ecf86381d85992b7e39; no voters:
I20260619 20:33:24.496831 15676 leader_election.cc:288] T 00000000000000000000000000000000 P e0b1de5761db4ecf86381d85992b7e39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:33:24.496881 15681 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0b1de5761db4ecf86381d85992b7e39 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:33:24.497228 15676 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0b1de5761db4ecf86381d85992b7e39 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:33:24.497937 15681 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0b1de5761db4ecf86381d85992b7e39 [term 1 LEADER]: Becoming Leader. State: Replica: e0b1de5761db4ecf86381d85992b7e39, State: Running, Role: LEADER
I20260619 20:33:24.498049 15681 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0b1de5761db4ecf86381d85992b7e39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0b1de5761db4ecf86381d85992b7e39" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35845 } }
I20260619 20:33:24.498741 15681 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0b1de5761db4ecf86381d85992b7e39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0b1de5761db4ecf86381d85992b7e39. Latest consensus state: current_term: 1 leader_uuid: "e0b1de5761db4ecf86381d85992b7e39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0b1de5761db4ecf86381d85992b7e39" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35845 } } }
I20260619 20:33:24.499492 15681 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0b1de5761db4ecf86381d85992b7e39 [sys.catalog]: This master's current role is: LEADER
I20260619 20:33:24.499821 15682 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0b1de5761db4ecf86381d85992b7e39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0b1de5761db4ecf86381d85992b7e39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0b1de5761db4ecf86381d85992b7e39" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35845 } } }
I20260619 20:33:24.499925 15682 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0b1de5761db4ecf86381d85992b7e39 [sys.catalog]: This master's current role is: LEADER
I20260619 20:33:24.501005 15676 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 20:33:24.501405 15690 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Jun 19 20:33:24 dist-test-slave-4m58 krb5kdc[15164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:33:24.502441 15690 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:33:24.510423 15680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:33:24.492870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37246 (local address 127.4.83.126:35845)
0619 20:33:24.493064 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 20:33:24.493073 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:33:24.493100 (+ 27us) server_negotiation.cc:407] Connection header received
0619 20:33:24.494826 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:33:24.494836 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:33:24.494906 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:33:24.495075 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:33:24.495896 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:33:24.497021 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:33:24.497882 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:33:24.498857 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:33:24.499204 (+ 347us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:33:24.502486 (+ 3282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:33:24.502491 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:33:24.502491 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:33:24.502506 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:33:24.508428 (+ 5922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:33:24.508829 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:33:24.508834 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:33:24.508834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:33:24.508965 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:33:24.509151 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:33:24.509152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:33:24.509152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:33:24.509538 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:33:24.509692 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 20:33:24.509815 (+ 123us) server_negotiation.cc:299] Negotiation successful
0619 20:33:24.510104 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":82,"threads_started":1}
I20260619 20:33:24.512327 15690 catalog_manager.cc:1347] Generated new cluster ID: 5f71a20bb77c47ca8b24d98a18687f31
I20260619 20:33:24.512389 15690 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:33:24.526995 15690 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:33:24.527923 15690 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:33:24.533317 15690 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0b1de5761db4ecf86381d85992b7e39: Generated new TSK 0
I20260619 20:33:24.533700 15690 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.83.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
I20260619 20:33:25.193131 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.83.84:38361/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:0
--local_ip_for_outbound_sockets=127.4.83.65
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:35845
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:42615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:33:25.282289 15721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:33:25.282518 15721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:33:25.282553 15721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:33:25.283806 15721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:33:25.283880 15721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:33:25.283921 15721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:33:25.285502 15721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:42615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.4.83.84:38361/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:35845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:33:25.285724 15721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:33:25.285944 15721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:33:25.290925 15726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:33:25.291079 15729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:33:25.291112 15721 server_base.cc:1034] running on GCE node
W20260619 20:33:25.290941 15727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:33:25.291385 15721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:33:25.291633 15721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:33:25.292804 15721 hybrid_clock.cc:648] HybridClock initialized: now 1781901205292771 us; error 51 us; skew 500 ppm
Jun 19 20:33:25 dist-test-slave-4m58 krb5kdc[15164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901205, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:33:25.295624 15721 init.cc:377] Logged in from keytab as kudu/127.4.83.65@KRBTEST.COM (short username kudu)
I20260619 20:33:25.296339 15721 webserver.cc:460] Webserver started at http://127.4.83.65:39885/ using document root <none> and password file <none>
I20260619 20:33:25.296612 15721 fs_manager.cc:359] Metadata directory not provided
I20260619 20:33:25.296681 15721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:33:25.296850 15721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 20:33:25 dist-test-slave-4m58 krb5kdc[15164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901205, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for HTTP/127.4.83.84@KRBTEST.COM
I20260619 20:33:25.313338 15721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-0/data/instance:
uuid: "9bc25f1fb476449ba2adf2b99ba83970"
format_stamp: "Formatted at 2026-06-19 20:33:25 on dist-test-slave-4m58"
server_key: "96309c10edccf84bd3530bd8c327581c"
server_key_iv: "d7bf225eac6a643320fdd8f511ed68ac"
server_key_version: "kuduclusterkey@0"
I20260619 20:33:25.313736 15721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance:
uuid: "9bc25f1fb476449ba2adf2b99ba83970"
format_stamp: "Formatted at 2026-06-19 20:33:25 on dist-test-slave-4m58"
server_key: "96309c10edccf84bd3530bd8c327581c"
server_key_iv: "d7bf225eac6a643320fdd8f511ed68ac"
server_key_version: "kuduclusterkey@0"
I20260619 20:33:25.320295 15721 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.000s
I20260619 20:33:25.333460 15736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:33:25.333686 15721 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:33:25.333781 15721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "9bc25f1fb476449ba2adf2b99ba83970"
format_stamp: "Formatted at 2026-06-19 20:33:25 on dist-test-slave-4m58"
server_key: "96309c10edccf84bd3530bd8c327581c"
server_key_iv: "d7bf225eac6a643320fdd8f511ed68ac"
server_key_version: "kuduclusterkey@0"
I20260619 20:33:25.333891 15721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:33:25.355710 15721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:33:25.358095 15721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:33:25.358299 15721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:33:25.358549 15721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:33:25.358857 15721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:33:25.358891 15721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:33:25.358919 15721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:33:25.358945 15721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:33:25.365520 15721 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:34729
I20260619 20:33:25.365939 15721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
I20260619 20:33:25.368970 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 15721
I20260619 20:33:25.369084 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance
I20260619 20:33:25.369293 4429 external_mini_cluster.cc:1428] Setting key bc1ab63ac7e6d261f97921f2e90d7236
I20260619 20:33:25.371872 15849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:34729 every 8 connection(s)
Jun 19 20:33:25 dist-test-slave-4m58 krb5kdc[15164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901205, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:33:25.376037 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:33:25.366666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:57417 (local address 127.4.83.126:35845)
0619 20:33:25.366810 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 20:33:25.366812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:33:25.366997 (+ 185us) server_negotiation.cc:407] Connection header received
0619 20:33:25.367528 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:33:25.367530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:33:25.367545 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:33:25.367608 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:33:25.368186 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.368849 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.370061 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.370263 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.370319 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:33:25.372650 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:33:25.372652 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:33:25.372652 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:33:25.372660 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:33:25.374394 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:33:25.374834 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:33:25.374835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:33:25.374836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:33:25.374907 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:33:25.375231 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:33:25.375232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:33:25.375232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:33:25.375379 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:33:25.375428 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 20:33:25.375861 (+ 433us) server_negotiation.cc:299] Negotiation successful
0619 20:33:25.375955 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":81,"threads_started":1}
I20260619 20:33:25.376521 15853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:33:25.366650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35845 (local address 127.4.83.65:57417)
0619 20:33:25.366871 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0619 20:33:25.366913 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 20:33:25.367433 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:33:25.367676 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:33:25.367680 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:33:25.367903 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:33:25.368103 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:33:25.368107 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.368963 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.368965 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:33:25.369978 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:33:25.369980 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.370334 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.370335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:33:25.370472 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:33:25.370957 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:33:25.370965 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:33:25.372566 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:33:25.374478 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:33:25.374486 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:33:25.374488 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:33:25.374706 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:33:25.374986 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:33:25.374987 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:33:25.374988 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:33:25.375130 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:33:25.375476 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:33:25.375483 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:33:25.375669 (+ 186us) client_negotiation.cc:769] Sending connection context
0619 20:33:25.376171 (+ 502us) client_negotiation.cc:240] Negotiation successful
0619 20:33:25.376351 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":88,"threads_started":1}
I20260619 20:33:25.376988 15850 heartbeater.cc:347] Connected to a master server at 127.4.83.126:35845
I20260619 20:33:25.377084 15850 heartbeater.cc:464] Registering TS with master...
I20260619 20:33:25.377264 15850 heartbeater.cc:511] Master 127.4.83.126:35845 requested a full tablet report, sending...
I20260619 20:33:25.377727 15640 ts_manager.cc:194] Registered new tserver with Master: 9bc25f1fb476449ba2adf2b99ba83970 (127.4.83.65:34729)
I20260619 20:33:25.378301 15640 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.65@KRBTEST.COM'} at 127.4.83.65:57417
WARNING: no policy specified for kudu/127.4.83.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
I20260619 20:33:25.398942 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.83.84:38361/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:0
--local_ip_for_outbound_sockets=127.4.83.66
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:35845
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:42615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:33:25.505244 15858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:33:25.505537 15858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:33:25.505581 15858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:33:25.507391 15858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:33:25.507460 15858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:33:25.507500 15858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:33:25.509769 15858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:42615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.4.83.84:38361/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:35845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:33:25.510046 15858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:33:25.510330 15858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:33:25.517407 15868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:33:25.517416 15866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:33:25.517792 15865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:33:25.518391 15858 server_base.cc:1034] running on GCE node
I20260619 20:33:25.518648 15858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:33:25.518935 15858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:33:25.520102 15858 hybrid_clock.cc:648] HybridClock initialized: now 1781901205520069 us; error 46 us; skew 500 ppm
Jun 19 20:33:25 dist-test-slave-4m58 krb5kdc[15164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901205, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:33:25.523483 15858 init.cc:377] Logged in from keytab as kudu/127.4.83.66@KRBTEST.COM (short username kudu)
I20260619 20:33:25.524133 15858 webserver.cc:460] Webserver started at http://127.4.83.66:34675/ using document root <none> and password file <none>
I20260619 20:33:25.524396 15858 fs_manager.cc:359] Metadata directory not provided
I20260619 20:33:25.524452 15858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:33:25.524621 15858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 20:33:25 dist-test-slave-4m58 krb5kdc[15164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901205, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for HTTP/127.4.83.84@KRBTEST.COM
I20260619 20:33:25.539400 15858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-1/data/instance:
uuid: "01771fe84dfe42ec857086bccf978072"
format_stamp: "Formatted at 2026-06-19 20:33:25 on dist-test-slave-4m58"
server_key: "e3b6bfbaafda74886a9b8a0393bd959c"
server_key_iv: "9a977c45cb9a61cb044f1053ae746e5f"
server_key_version: "kuduclusterkey@0"
I20260619 20:33:25.540098 15858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance:
uuid: "01771fe84dfe42ec857086bccf978072"
format_stamp: "Formatted at 2026-06-19 20:33:25 on dist-test-slave-4m58"
server_key: "e3b6bfbaafda74886a9b8a0393bd959c"
server_key_iv: "9a977c45cb9a61cb044f1053ae746e5f"
server_key_version: "kuduclusterkey@0"
I20260619 20:33:25.548108 15858 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.000s sys 0.001s
Jun 19 20:33:25 dist-test-slave-4m58 krb5kdc[15164](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781901205, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:33:25.563906 15875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:33:25.564079 15858 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:33:25.564173 15858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "01771fe84dfe42ec857086bccf978072"
format_stamp: "Formatted at 2026-06-19 20:33:25 on dist-test-slave-4m58"
server_key: "e3b6bfbaafda74886a9b8a0393bd959c"
server_key_iv: "9a977c45cb9a61cb044f1053ae746e5f"
server_key_version: "kuduclusterkey@0"
I20260619 20:33:25.564275 15858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:33:25.589267 15858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:33:25.591645 15858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:33:25.591806 15858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:33:25.592051 15858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:33:25.592417 15858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:33:25.592460 15858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:33:25.592497 15858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:33:25.592514 15858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:33:25.599255 15858 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:34057
I20260619 20:33:25.599516 15988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:34057 every 8 connection(s)
I20260619 20:33:25.599705 15858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
Jun 19 20:33:25 dist-test-slave-4m58 krb5kdc[15164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901205, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:33:25.609078 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 15858
I20260619 20:33:25.609228 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance
I20260619 20:33:25.609407 4429 external_mini_cluster.cc:1428] Setting key c99c959085f05ea240b1a029b997bfb6
I20260619 20:33:25.612736 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:33:25.601339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:53995 (local address 127.4.83.126:35845)
0619 20:33:25.601390 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 20:33:25.601392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:33:25.601736 (+ 344us) server_negotiation.cc:407] Connection header received
0619 20:33:25.602818 (+ 1082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:33:25.602821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:33:25.602839 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:33:25.602874 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:33:25.603859 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.604378 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.605444 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.605664 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.605722 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:33:25.608372 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:33:25.608375 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:33:25.608375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:33:25.608384 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:33:25.610064 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:33:25.611700 (+ 1636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:33:25.611702 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:33:25.611703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:33:25.611770 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:33:25.612033 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:33:25.612034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:33:25.612034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:33:25.612167 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:33:25.612227 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 20:33:25.612524 (+ 297us) server_negotiation.cc:299] Negotiation successful
0619 20:33:25.612634 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:33:25.612946 15991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:33:25.601397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35845 (local address 127.4.83.66:53995)
0619 20:33:25.601587 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 20:33:25.601638 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 20:33:25.602587 (+ 949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:33:25.603276 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:33:25.603281 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:33:25.603496 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:33:25.603708 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:33:25.603712 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.604471 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.604473 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:33:25.605346 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:33:25.605349 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.605739 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.605740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:33:25.605863 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:33:25.606470 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:33:25.606480 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:33:25.608257 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:33:25.611324 (+ 3067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:33:25.611334 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:33:25.611336 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:33:25.611608 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:33:25.611850 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:33:25.611851 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:33:25.611852 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:33:25.611972 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:33:25.612224 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:33:25.612231 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:33:25.612399 (+ 168us) client_negotiation.cc:769] Sending connection context
0619 20:33:25.612495 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 20:33:25.612660 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":88,"threads_started":1}
I20260619 20:33:25.613415 15989 heartbeater.cc:347] Connected to a master server at 127.4.83.126:35845
I20260619 20:33:25.613513 15989 heartbeater.cc:464] Registering TS with master...
I20260619 20:33:25.613708 15989 heartbeater.cc:511] Master 127.4.83.126:35845 requested a full tablet report, sending...
I20260619 20:33:25.614073 15640 ts_manager.cc:194] Registered new tserver with Master: 01771fe84dfe42ec857086bccf978072 (127.4.83.66:34057)
I20260619 20:33:25.614434 15640 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.66@KRBTEST.COM'} at 127.4.83.66:53995
WARNING: no policy specified for kudu/127.4.83.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
I20260619 20:33:25.639938 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.83.84:38361/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:0
--local_ip_for_outbound_sockets=127.4.83.67
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:35845
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:42615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:33:25.761102 15995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:33:25.761428 15995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:33:25.761473 15995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:33:25.763430 15995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:33:25.763504 15995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:33:25.763549 15995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:33:25.765868 15995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:42615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.4.83.84:38361/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:35845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:33:25.766139 15995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:33:25.766446 15995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:33:25.772820 16001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:33:25.772841 16003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:33:25.773257 16000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:33:25.773794 15995 server_base.cc:1034] running on GCE node
I20260619 20:33:25.774024 15995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:33:25.774295 15995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:33:25.775457 15995 hybrid_clock.cc:648] HybridClock initialized: now 1781901205775437 us; error 33 us; skew 500 ppm
Jun 19 20:33:25 dist-test-slave-4m58 krb5kdc[15164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901205, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:33:25.778699 15995 init.cc:377] Logged in from keytab as kudu/127.4.83.67@KRBTEST.COM (short username kudu)
I20260619 20:33:25.779255 15995 webserver.cc:460] Webserver started at http://127.4.83.67:35315/ using document root <none> and password file <none>
I20260619 20:33:25.779516 15995 fs_manager.cc:359] Metadata directory not provided
I20260619 20:33:25.779588 15995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:33:25.779731 15995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 20:33:25 dist-test-slave-4m58 krb5kdc[15164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901205, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for HTTP/127.4.83.84@KRBTEST.COM
I20260619 20:33:25.793655 15995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-2/data/instance:
uuid: "1bb5518f4c9146be917938777c0dda03"
format_stamp: "Formatted at 2026-06-19 20:33:25 on dist-test-slave-4m58"
server_key: "454165deb40ebc956f0bb910dd306549"
server_key_iv: "bd3cf2c450522293035e47c9b28e8317"
server_key_version: "kuduclusterkey@0"
I20260619 20:33:25.794277 15995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance:
uuid: "1bb5518f4c9146be917938777c0dda03"
format_stamp: "Formatted at 2026-06-19 20:33:25 on dist-test-slave-4m58"
server_key: "454165deb40ebc956f0bb910dd306549"
server_key_iv: "bd3cf2c450522293035e47c9b28e8317"
server_key_version: "kuduclusterkey@0"
I20260619 20:33:25.801541 15995 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.000s
I20260619 20:33:25.819991 16010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:33:25.820202 15995 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.000s
I20260619 20:33:25.820299 15995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "1bb5518f4c9146be917938777c0dda03"
format_stamp: "Formatted at 2026-06-19 20:33:25 on dist-test-slave-4m58"
server_key: "454165deb40ebc956f0bb910dd306549"
server_key_iv: "bd3cf2c450522293035e47c9b28e8317"
server_key_version: "kuduclusterkey@0"
I20260619 20:33:25.820408 15995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:33:25.833293 15995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:33:25.835204 15995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:33:25.835347 15995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:33:25.835534 15995 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:33:25.835867 15995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:33:25.835912 15995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:33:25.835945 15995 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:33:25.835965 15995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:33:25.840773 15995 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:40183
I20260619 20:33:25.841202 15995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
I20260619 20:33:25.843283 16123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:40183 every 8 connection(s)
Jun 19 20:33:25 dist-test-slave-4m58 krb5kdc[15164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901205, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:33:25.850226 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 15995
I20260619 20:33:25.850343 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance
I20260619 20:33:25.850520 4429 external_mini_cluster.cc:1428] Setting key 6f6b4ff49e2496bf4521933af71a4f63
I20260619 20:33:25.853368 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:33:25.843935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:49559 (local address 127.4.83.126:35845)
0619 20:33:25.844008 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 20:33:25.844011 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:33:25.844252 (+ 241us) server_negotiation.cc:407] Connection header received
0619 20:33:25.844912 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:33:25.844913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:33:25.844923 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:33:25.844973 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:33:25.845717 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.846257 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.847745 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.847991 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.848043 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:33:25.850330 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:33:25.850332 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:33:25.850332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:33:25.850341 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:33:25.851868 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:33:25.852370 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:33:25.852370 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:33:25.852371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:33:25.852418 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:33:25.852710 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:33:25.852710 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:33:25.852711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:33:25.852831 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:33:25.852876 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 20:33:25.853239 (+ 363us) server_negotiation.cc:299] Negotiation successful
0619 20:33:25.853303 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 20:33:25.853703 16126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:33:25.843903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35845 (local address 127.4.83.67:49559)
0619 20:33:25.844096 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 20:33:25.844154 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 20:33:25.844792 (+ 638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:33:25.845063 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:33:25.845068 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:33:25.845318 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:33:25.845612 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:33:25.845619 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.846363 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.846364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:33:25.847656 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:33:25.847660 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.848064 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.848066 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:33:25.848216 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:33:25.848741 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:33:25.848750 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:33:25.850185 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:33:25.851939 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:33:25.851949 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:33:25.851953 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:33:25.852296 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:33:25.852472 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:33:25.852472 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:33:25.852473 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:33:25.852650 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:33:25.852885 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:33:25.852891 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:33:25.853103 (+ 212us) client_negotiation.cc:769] Sending connection context
0619 20:33:25.853231 (+ 128us) client_negotiation.cc:240] Negotiation successful
0619 20:33:25.853417 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":96,"threads_started":1}
I20260619 20:33:25.854179 16124 heartbeater.cc:347] Connected to a master server at 127.4.83.126:35845
I20260619 20:33:25.854276 16124 heartbeater.cc:464] Registering TS with master...
I20260619 20:33:25.854455 16124 heartbeater.cc:511] Master 127.4.83.126:35845 requested a full tablet report, sending...
I20260619 20:33:25.854851 15640 ts_manager.cc:194] Registered new tserver with Master: 1bb5518f4c9146be917938777c0dda03 (127.4.83.67:40183)
I20260619 20:33:25.855211 15640 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.67@KRBTEST.COM'} at 127.4.83.67:49559
I20260619 20:33:25.864306 4429 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:33:25.873983 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:33:25.865766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37258 (local address 127.4.83.126:35845)
0619 20:33:25.865808 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 20:33:25.865811 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:33:25.866053 (+ 242us) server_negotiation.cc:407] Connection header received
0619 20:33:25.866157 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:33:25.866158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:33:25.866168 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:33:25.866198 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:33:25.866939 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.867759 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.868881 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.869219 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.869321 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:33:25.870823 (+ 1502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:33:25.870825 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:33:25.870826 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:33:25.870834 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:33:25.872491 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:33:25.872915 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:33:25.872917 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:33:25.872917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:33:25.872976 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:33:25.873273 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:33:25.873274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:33:25.873274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:33:25.873429 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:33:25.873505 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 20:33:25.873749 (+ 244us) server_negotiation.cc:299] Negotiation successful
0619 20:33:25.873834 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:33:25.876457 15640 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37258:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:33:25.878836 15640 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:33:25.886996 16139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:33:25.881549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:34729 (local address 127.0.0.1:39364)
0619 20:33:25.882339 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0619 20:33:25.882381 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 20:33:25.882514 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:33:25.882825 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:33:25.882829 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:33:25.882831 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:33:25.882902 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:33:25.882904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.884292 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.884293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:33:25.885633 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:33:25.885635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.886688 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.886688 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:33:25.886772 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:33:25.886786 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 20:33:25.886852 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 20:33:25.886908 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":84,"threads_started":1}
I20260619 20:33:25.887454 16136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:33:25.881559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:40183 (local address 127.0.0.1:57618)
0619 20:33:25.882646 (+ 1087us) negotiation.cc:107] Waiting for socket to connect
0619 20:33:25.882656 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:33:25.882692 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:33:25.883108 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:33:25.883108 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:33:25.883111 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:33:25.883155 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:33:25.883156 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.884740 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.884741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:33:25.885838 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:33:25.885842 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.887198 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.887199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:33:25.887265 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:33:25.887269 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:33:25.887333 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 20:33:25.887376 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1060,"thread_start_us":90,"threads_started":1}
I20260619 20:33:25.887809 16141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:33:25.881979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39364 (local address 127.4.83.65:34729)
0619 20:33:25.882116 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 20:33:25.882121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:33:25.882471 (+ 350us) server_negotiation.cc:407] Connection header received
0619 20:33:25.882622 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:33:25.882625 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:33:25.882644 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:33:25.882741 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:33:25.882984 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.884198 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.885733 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.886525 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.886593 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:33:25.886653 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 20:33:25.887567 (+ 914us) server_negotiation.cc:299] Negotiation successful
0619 20:33:25.887615 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":70,"threads_started":1}
I20260619 20:33:25.888031 16138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:33:25.881715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57618 (local address 127.4.83.67:40183)
0619 20:33:25.882897 (+ 1182us) server_negotiation.cc:206] Beginning negotiation
0619 20:33:25.882902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:33:25.882910 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:33:25.882945 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:33:25.882947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:33:25.882961 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:33:25.883030 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:33:25.883224 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.884649 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.885955 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.887023 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.887091 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:33:25.887162 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 20:33:25.887905 (+ 743us) server_negotiation.cc:299] Negotiation successful
0619 20:33:25.887950 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1148,"thread_start_us":72,"threads_started":1}
I20260619 20:33:25.888590 15784 tablet_service.cc:1467] Processing CreateTablet for tablet 4901140a01d94bcebf93aa3dd250ce1b (DEFAULT_TABLE table=test-table [id=16acdad53ffb448db422e8ffddb68ef2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:33:25.888652 16058 tablet_service.cc:1467] Processing CreateTablet for tablet 4901140a01d94bcebf93aa3dd250ce1b (DEFAULT_TABLE table=test-table [id=16acdad53ffb448db422e8ffddb68ef2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:33:25.888954 15784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4901140a01d94bcebf93aa3dd250ce1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:33:25.888972 16058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4901140a01d94bcebf93aa3dd250ce1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:33:25.892820 16142 tablet_bootstrap.cc:492] T 4901140a01d94bcebf93aa3dd250ce1b P 9bc25f1fb476449ba2adf2b99ba83970: Bootstrap starting.
I20260619 20:33:25.893524 16142 tablet_bootstrap.cc:654] T 4901140a01d94bcebf93aa3dd250ce1b P 9bc25f1fb476449ba2adf2b99ba83970: Neither blocks nor log segments found. Creating new log.
I20260619 20:33:25.893862 16142 log.cc:824] T 4901140a01d94bcebf93aa3dd250ce1b P 9bc25f1fb476449ba2adf2b99ba83970: Log is configured to *not* fsync() on all Append() calls
I20260619 20:33:25.894193 16143 tablet_bootstrap.cc:492] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03: Bootstrap starting.
I20260619 20:33:25.895121 16143 tablet_bootstrap.cc:654] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03: Neither blocks nor log segments found. Creating new log.
I20260619 20:33:25.896134 16143 log.cc:824] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03: Log is configured to *not* fsync() on all Append() calls
I20260619 20:33:25.896713 16142 tablet_bootstrap.cc:492] T 4901140a01d94bcebf93aa3dd250ce1b P 9bc25f1fb476449ba2adf2b99ba83970: No bootstrap required, opened a new log
I20260619 20:33:25.896791 16142 ts_tablet_manager.cc:1397] T 4901140a01d94bcebf93aa3dd250ce1b P 9bc25f1fb476449ba2adf2b99ba83970: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 20:33:25.898443 16137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:33:25.881794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:34057 (local address 127.0.0.1:53752)
0619 20:33:25.882755 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0619 20:33:25.882762 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0619 20:33:25.882794 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:33:25.887253 (+ 4459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:33:25.887254 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:33:25.887256 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:33:25.887292 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:33:25.887293 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.888506 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.888507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:33:25.889498 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:33:25.889500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.898158 (+ 8658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.898160 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:33:25.898237 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:33:25.898245 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 20:33:25.898277 (+ 32us) client_negotiation.cc:240] Negotiation successful
0619 20:33:25.898364 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":950,"thread_start_us":66,"threads_started":1}
I20260619 20:33:25.898952 16140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:33:25.881903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53752 (local address 127.4.83.66:34057)
0619 20:33:25.887015 (+ 5112us) server_negotiation.cc:206] Beginning negotiation
0619 20:33:25.887020 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:33:25.887031 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:33:25.887071 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:33:25.887073 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:33:25.887089 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:33:25.887172 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:33:25.887361 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.888393 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.897363 (+ 8970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.898045 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.898764 (+ 719us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:33:25.898833 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 20:33:25.898858 (+ 25us) server_negotiation.cc:299] Negotiation successful
0619 20:33:25.898893 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5067,"thread_start_us":62,"threads_started":1}
I20260619 20:33:25.899115 16143 tablet_bootstrap.cc:492] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03: No bootstrap required, opened a new log
I20260619 20:33:25.899190 16143 ts_tablet_manager.cc:1397] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260619 20:33:25.899624 15918 tablet_service.cc:1467] Processing CreateTablet for tablet 4901140a01d94bcebf93aa3dd250ce1b (DEFAULT_TABLE table=test-table [id=16acdad53ffb448db422e8ffddb68ef2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:33:25.899972 15918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4901140a01d94bcebf93aa3dd250ce1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:33:25.902452 16143 raft_consensus.cc:348] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc25f1fb476449ba2adf2b99ba83970" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34729 } } peers { permanent_uuid: "01771fe84dfe42ec857086bccf978072" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34057 } } peers { permanent_uuid: "1bb5518f4c9146be917938777c0dda03" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40183 } }
I20260619 20:33:25.902624 16143 raft_consensus.cc:374] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:33:25.902670 16143 raft_consensus.cc:729] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bb5518f4c9146be917938777c0dda03, State: Initialized, Role: FOLLOWER
I20260619 20:33:25.903074 16142 raft_consensus.cc:348] T 4901140a01d94bcebf93aa3dd250ce1b P 9bc25f1fb476449ba2adf2b99ba83970 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc25f1fb476449ba2adf2b99ba83970" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34729 } } peers { permanent_uuid: "01771fe84dfe42ec857086bccf978072" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34057 } } peers { permanent_uuid: "1bb5518f4c9146be917938777c0dda03" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40183 } }
I20260619 20:33:25.903214 16142 raft_consensus.cc:374] T 4901140a01d94bcebf93aa3dd250ce1b P 9bc25f1fb476449ba2adf2b99ba83970 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:33:25.903246 16142 raft_consensus.cc:729] T 4901140a01d94bcebf93aa3dd250ce1b P 9bc25f1fb476449ba2adf2b99ba83970 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bc25f1fb476449ba2adf2b99ba83970, State: Initialized, Role: FOLLOWER
I20260619 20:33:25.903398 16143 consensus_queue.cc:260] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc25f1fb476449ba2adf2b99ba83970" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34729 } } peers { permanent_uuid: "01771fe84dfe42ec857086bccf978072" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34057 } } peers { permanent_uuid: "1bb5518f4c9146be917938777c0dda03" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40183 } }
I20260619 20:33:25.903682 16143 ts_tablet_manager.cc:1428] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260619 20:33:25.903808 16142 consensus_queue.cc:260] T 4901140a01d94bcebf93aa3dd250ce1b P 9bc25f1fb476449ba2adf2b99ba83970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc25f1fb476449ba2adf2b99ba83970" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34729 } } peers { permanent_uuid: "01771fe84dfe42ec857086bccf978072" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34057 } } peers { permanent_uuid: "1bb5518f4c9146be917938777c0dda03" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40183 } }
I20260619 20:33:25.904090 16142 ts_tablet_manager.cc:1428] T 4901140a01d94bcebf93aa3dd250ce1b P 9bc25f1fb476449ba2adf2b99ba83970: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260619 20:33:25.904217 16124 heartbeater.cc:503] Master 127.4.83.126:35845 was elected leader, sending a full tablet report...
I20260619 20:33:25.905254 15850 heartbeater.cc:503] Master 127.4.83.126:35845 was elected leader, sending a full tablet report...
I20260619 20:33:25.906083 16148 tablet_bootstrap.cc:492] T 4901140a01d94bcebf93aa3dd250ce1b P 01771fe84dfe42ec857086bccf978072: Bootstrap starting.
I20260619 20:33:25.907094 16148 tablet_bootstrap.cc:654] T 4901140a01d94bcebf93aa3dd250ce1b P 01771fe84dfe42ec857086bccf978072: Neither blocks nor log segments found. Creating new log.
I20260619 20:33:25.907421 16148 log.cc:824] T 4901140a01d94bcebf93aa3dd250ce1b P 01771fe84dfe42ec857086bccf978072: Log is configured to *not* fsync() on all Append() calls
I20260619 20:33:25.908192 16148 tablet_bootstrap.cc:492] T 4901140a01d94bcebf93aa3dd250ce1b P 01771fe84dfe42ec857086bccf978072: No bootstrap required, opened a new log
I20260619 20:33:25.908267 16148 ts_tablet_manager.cc:1397] T 4901140a01d94bcebf93aa3dd250ce1b P 01771fe84dfe42ec857086bccf978072: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:33:25.910287 16148 raft_consensus.cc:348] T 4901140a01d94bcebf93aa3dd250ce1b P 01771fe84dfe42ec857086bccf978072 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc25f1fb476449ba2adf2b99ba83970" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34729 } } peers { permanent_uuid: "01771fe84dfe42ec857086bccf978072" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34057 } } peers { permanent_uuid: "1bb5518f4c9146be917938777c0dda03" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40183 } }
I20260619 20:33:25.910482 16148 raft_consensus.cc:374] T 4901140a01d94bcebf93aa3dd250ce1b P 01771fe84dfe42ec857086bccf978072 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:33:25.910516 16148 raft_consensus.cc:729] T 4901140a01d94bcebf93aa3dd250ce1b P 01771fe84dfe42ec857086bccf978072 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01771fe84dfe42ec857086bccf978072, State: Initialized, Role: FOLLOWER
I20260619 20:33:25.910627 16148 consensus_queue.cc:260] T 4901140a01d94bcebf93aa3dd250ce1b P 01771fe84dfe42ec857086bccf978072 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc25f1fb476449ba2adf2b99ba83970" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34729 } } peers { permanent_uuid: "01771fe84dfe42ec857086bccf978072" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34057 } } peers { permanent_uuid: "1bb5518f4c9146be917938777c0dda03" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40183 } }
I20260619 20:33:25.910872 16148 ts_tablet_manager.cc:1428] T 4901140a01d94bcebf93aa3dd250ce1b P 01771fe84dfe42ec857086bccf978072: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:33:25.911007 15989 heartbeater.cc:503] Master 127.4.83.126:35845 was elected leader, sending a full tablet report...
I20260619 20:33:25.960340 16146 raft_consensus.cc:482] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:33:25.960500 16146 raft_consensus.cc:504] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc25f1fb476449ba2adf2b99ba83970" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34729 } } peers { permanent_uuid: "01771fe84dfe42ec857086bccf978072" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34057 } } peers { permanent_uuid: "1bb5518f4c9146be917938777c0dda03" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40183 } }
I20260619 20:33:25.960898 16146 leader_election.cc:288] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9bc25f1fb476449ba2adf2b99ba83970 (127.4.83.65:34729), 01771fe84dfe42ec857086bccf978072 (127.4.83.66:34057)
I20260619 20:33:25.964975 16126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:33:25.961071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:34729 (local address 127.4.83.67:51541)
0619 20:33:25.961173 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0619 20:33:25.961184 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:33:25.961225 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:33:25.961558 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:33:25.961559 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:33:25.961561 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:33:25.961629 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:33:25.961632 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.962638 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.962639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:33:25.963885 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:33:25.963888 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.964713 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.964714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:33:25.964784 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:33:25.964788 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:33:25.964833 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 20:33:25.964878 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"mutex_wait_us":27}
I20260619 20:33:25.965040 16140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:33:25.961339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:43895 (local address 127.4.83.66:34057)
0619 20:33:25.961553 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0619 20:33:25.961555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:33:25.961561 (+ 6us) server_negotiation.cc:407] Connection header received
0619 20:33:25.961583 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:33:25.961584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:33:25.961594 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:33:25.961635 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:33:25.961843 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.962580 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.964012 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.964623 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.964667 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:33:25.964678 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 20:33:25.964928 (+ 250us) server_negotiation.cc:299] Negotiation successful
0619 20:33:25.964971 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194}
I20260619 20:33:25.965291 16151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:33:25.961088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:34057 (local address 127.4.83.67:43895)
0619 20:33:25.961433 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0619 20:33:25.961441 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:33:25.961484 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:33:25.961681 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:33:25.961682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:33:25.961684 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:33:25.961762 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:33:25.961764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.962697 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.962698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:33:25.963916 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:33:25.963918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.964778 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.964779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:33:25.964854 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:33:25.964857 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:33:25.965148 (+ 291us) client_negotiation.cc:240] Negotiation successful
0619 20:33:25.965188 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":114,"threads_started":1}
I20260619 20:33:25.965585 15935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4901140a01d94bcebf93aa3dd250ce1b" candidate_uuid: "1bb5518f4c9146be917938777c0dda03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01771fe84dfe42ec857086bccf978072" is_pre_election: true
I20260619 20:33:25.965688 16141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:33:25.961314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:51541 (local address 127.4.83.65:34729)
0619 20:33:25.961392 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 20:33:25.961394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:33:25.961403 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:33:25.961431 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:33:25.961433 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:33:25.961446 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:33:25.961500 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:33:25.961711 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.962500 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.963990 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.964592 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.964670 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:33:25.964681 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 20:33:25.965561 (+ 880us) server_negotiation.cc:299] Negotiation successful
0619 20:33:25.965611 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 20:33:25.965741 15935 raft_consensus.cc:2454] T 4901140a01d94bcebf93aa3dd250ce1b P 01771fe84dfe42ec857086bccf978072 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bb5518f4c9146be917938777c0dda03 in term 0.
I20260619 20:33:25.965927 15788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4901140a01d94bcebf93aa3dd250ce1b" candidate_uuid: "1bb5518f4c9146be917938777c0dda03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bc25f1fb476449ba2adf2b99ba83970" is_pre_election: true
I20260619 20:33:25.966058 15788 raft_consensus.cc:2454] T 4901140a01d94bcebf93aa3dd250ce1b P 9bc25f1fb476449ba2adf2b99ba83970 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bb5518f4c9146be917938777c0dda03 in term 0.
I20260619 20:33:25.966058 16012 leader_election.cc:302] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01771fe84dfe42ec857086bccf978072, 1bb5518f4c9146be917938777c0dda03; no voters:
I20260619 20:33:25.966176 16146 raft_consensus.cc:2781] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:33:25.966225 16146 raft_consensus.cc:482] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:33:25.966248 16146 raft_consensus.cc:3037] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:33:25.967051 16146 raft_consensus.cc:504] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc25f1fb476449ba2adf2b99ba83970" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34729 } } peers { permanent_uuid: "01771fe84dfe42ec857086bccf978072" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34057 } } peers { permanent_uuid: "1bb5518f4c9146be917938777c0dda03" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40183 } }
I20260619 20:33:25.967187 16146 leader_election.cc:288] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03 [CANDIDATE]: Term 1 election: Requested vote from peers 9bc25f1fb476449ba2adf2b99ba83970 (127.4.83.65:34729), 01771fe84dfe42ec857086bccf978072 (127.4.83.66:34057)
I20260619 20:33:25.967353 15788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4901140a01d94bcebf93aa3dd250ce1b" candidate_uuid: "1bb5518f4c9146be917938777c0dda03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bc25f1fb476449ba2adf2b99ba83970"
I20260619 20:33:25.967382 15935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4901140a01d94bcebf93aa3dd250ce1b" candidate_uuid: "1bb5518f4c9146be917938777c0dda03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01771fe84dfe42ec857086bccf978072"
I20260619 20:33:25.967427 15788 raft_consensus.cc:3037] T 4901140a01d94bcebf93aa3dd250ce1b P 9bc25f1fb476449ba2adf2b99ba83970 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:33:25.967447 15935 raft_consensus.cc:3037] T 4901140a01d94bcebf93aa3dd250ce1b P 01771fe84dfe42ec857086bccf978072 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:33:25.968303 15935 raft_consensus.cc:2454] T 4901140a01d94bcebf93aa3dd250ce1b P 01771fe84dfe42ec857086bccf978072 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bb5518f4c9146be917938777c0dda03 in term 1.
I20260619 20:33:25.968401 15788 raft_consensus.cc:2454] T 4901140a01d94bcebf93aa3dd250ce1b P 9bc25f1fb476449ba2adf2b99ba83970 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bb5518f4c9146be917938777c0dda03 in term 1.
I20260619 20:33:25.968650 16013 leader_election.cc:302] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bb5518f4c9146be917938777c0dda03, 9bc25f1fb476449ba2adf2b99ba83970; no voters:
I20260619 20:33:25.968859 16146 raft_consensus.cc:2781] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:33:25.969020 16146 raft_consensus.cc:686] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03 [term 1 LEADER]: Becoming Leader. State: Replica: 1bb5518f4c9146be917938777c0dda03, State: Running, Role: LEADER
I20260619 20:33:25.969120 16146 consensus_queue.cc:237] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc25f1fb476449ba2adf2b99ba83970" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34729 } } peers { permanent_uuid: "01771fe84dfe42ec857086bccf978072" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34057 } } peers { permanent_uuid: "1bb5518f4c9146be917938777c0dda03" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40183 } }
I20260619 20:33:25.970008 15640 catalog_manager.cc:5495] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bb5518f4c9146be917938777c0dda03 (127.4.83.67). New cstate: current_term: 1 leader_uuid: "1bb5518f4c9146be917938777c0dda03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bc25f1fb476449ba2adf2b99ba83970" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01771fe84dfe42ec857086bccf978072" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bb5518f4c9146be917938777c0dda03" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 40183 } health_report { overall_health: HEALTHY } } }
I20260619 20:33:25.984308 16138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:33:25.978581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57624 (local address 127.4.83.67:40183)
0619 20:33:25.978648 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 20:33:25.978650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:33:25.978658 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:33:25.978697 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:33:25.978698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:33:25.978720 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:33:25.978797 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:33:25.979440 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.980244 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.981375 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:33:25.981688 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:33:25.981742 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:33:25.983667 (+ 1925us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:33:25.983861 (+ 194us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:33:25.983947 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 20:33:25.984196 (+ 249us) server_negotiation.cc:299] Negotiation successful
0619 20:33:25.984219 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 20:33:25.988889 15935 raft_consensus.cc:1261] T 4901140a01d94bcebf93aa3dd250ce1b P 01771fe84dfe42ec857086bccf978072 [term 1 FOLLOWER]: Refusing update from remote peer 1bb5518f4c9146be917938777c0dda03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:33:25.989253 15788 raft_consensus.cc:1261] T 4901140a01d94bcebf93aa3dd250ce1b P 9bc25f1fb476449ba2adf2b99ba83970 [term 1 FOLLOWER]: Refusing update from remote peer 1bb5518f4c9146be917938777c0dda03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:33:25.989296 16152 consensus_queue.cc:1035] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01771fe84dfe42ec857086bccf978072" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:33:25.989655 16152 consensus_queue.cc:1035] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bc25f1fb476449ba2adf2b99ba83970" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Jun 19 20:33:25 dist-test-slave-4m58 krb5kdc[15164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901205, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for HTTP/127.4.83.84@KRBTEST.COM
I20260619 20:33:26.003311 16158 mvcc.cc:205] Tried to move back new op lower bound from 7298667339727732736 to 7298667339650174976. Current Snapshot: MvccSnapshot[applied={T|T < 7298667339727732736}]
I20260619 20:33:26.003908 15640 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:33:26.004019 15640 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:33:26.004742 15640 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0b1de5761db4ecf86381d85992b7e39: Sending DeleteTablet for 3 replicas of tablet 4901140a01d94bcebf93aa3dd250ce1b
I20260619 20:33:26.005210 16058 tablet_service.cc:1514] Processing DeleteTablet for tablet 4901140a01d94bcebf93aa3dd250ce1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:33:26 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:57618
I20260619 20:33:26.005213 15784 tablet_service.cc:1514] Processing DeleteTablet for tablet 4901140a01d94bcebf93aa3dd250ce1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:33:26 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:39364
I20260619 20:33:26.005694 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 15721
I20260619 20:33:26.005805 16156 mvcc.cc:205] Tried to move back new op lower bound from 7298667339727732736 to 7298667339650174976. Current Snapshot: MvccSnapshot[applied={T|T < 7298667339727732736}]
I20260619 20:33:26.007802 15918 tablet_service.cc:1514] Processing DeleteTablet for tablet 4901140a01d94bcebf93aa3dd250ce1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:33:26 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:53752
I20260619 20:33:26.016115 16165 tablet_replica.cc:330] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03: stopping tablet replica
I20260619 20:33:26.016346 16165 raft_consensus.cc:2229] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:33:26.016572 16165 raft_consensus.cc:2258] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:33:26.017143 16165 ts_tablet_manager.cc:1905] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 20:33:26.023100 15626 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.83.65:34729 (error 108)
I20260619 20:33:26.023553 16165 ts_tablet_manager.cc:1918] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:33:26.024179 16165 log.cc:1196] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901102012824-4429-0/minicluster-data/ts-2/wal/wals/4901140a01d94bcebf93aa3dd250ce1b
I20260619 20:33:26.025164 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 15858
I20260619 20:33:26.024729 16165 ts_tablet_manager.cc:1939] T 4901140a01d94bcebf93aa3dd250ce1b P 1bb5518f4c9146be917938777c0dda03: Deleting consensus metadata
I20260619 20:33:26.027573 15624 catalog_manager.cc:4841] TS 1bb5518f4c9146be917938777c0dda03 (127.4.83.67:40183): tablet 4901140a01d94bcebf93aa3dd250ce1b (table test-table [id=16acdad53ffb448db422e8ffddb68ef2]) successfully deleted
I20260619 20:33:26.036360 16137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:33:26.036189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:34729 (local address 127.0.0.1:39370)
0619 20:33:26.036242 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 20:33:26.036301 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.65:34729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260619 20:33:26.036527 15626 catalog_manager.cc:4584] TS 9bc25f1fb476449ba2adf2b99ba83970 (127.4.83.65:34729): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4901140a01d94bcebf93aa3dd250ce1b: Network error: Client connection negotiation failed: client connection to 127.4.83.65:34729: connect: Connection refused (error 111)
I20260619 20:33:26.039594 16137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:33:26.039482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:34057 (local address 127.0.0.1:53766)
0619 20:33:26.039530 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 20:33:26.039553 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.66:34057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 20:33:26.039637 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 15995
I20260619 20:33:26.047065 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 15608
I20260619 20:33:26.054215 4429 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 20:33:26.399812 4429 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 20:33:26.399971 4429 mini_ranger.cc:66] Stopping Ranger...
I20260619 20:33:26.736512 4429 mini_ranger.cc:68] Stopped Ranger
2026-06-19 20:33:26.736 UTC [15194] LOG: received smart shutdown request
2026-06-19 20:33:26.740 UTC [15194] LOG: background worker "logical replication launcher" (PID 15203) exited with exit code 1
2026-06-19 20:33:26.740 UTC [15198] LOG: shutting down
2026-06-19 20:33:26.895 UTC [15194] LOG: database system is shut down
2026-06-19T20:33:26Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50457 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:33:27 dist-test-slave-4m58 krb5kdc[16176](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:33:27 dist-test-slave-4m58 krb5kdc[16176](info): set up 2 sockets
Jun 19 20:33:27 dist-test-slave-4m58 krb5kdc[16176](info): commencing operation
krb5kdc: starting...
W20260619 20:33:27.988757 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.962s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:33:28 dist-test-slave-4m58 krb5kdc[16176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:33:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:33:28Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.4.83.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.4.83.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.4.83.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/rangeradmin_127.4.83.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.4.83.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/rangeradmin_127.4.83.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.4.83.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.4.83.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.4.83.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/rangerlookup_127.4.83.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.4.83.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/rangerlookup_127.4.83.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.83.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.4.83.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.83.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/HTTP_127.4.83.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.83.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/HTTP_127.4.83.84@KRBTEST.COM.keytab.
I20260619 20:33:28.179991 4429 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskPZCFKi/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/postgres -l logfile start
2026-06-19 20:33:30.764 UTC [16206] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 20:33:30.764 UTC [16206] LOG: listening on IPv4 address "127.4.83.84", port 44765
2026-06-19 20:33:30.769 UTC [16206] LOG: listening on Unix socket "/tmp/.s.PGSQL.44765"
2026-06-19 20:33:30.793 UTC [16209] LOG: database system was shut down at 2026-06-19 20:33:29 UTC
2026-06-19 20:33:30.797 UTC [16206] LOG: database system is ready to accept connections
I20260619 20:33:31.668466 4429 mini_postgres.cc:96] Postgres bound to 44765
2026-06-19 20:33:31.674 UTC [16217] FATAL: database "slave" does not exist
127.4.83.84:44765 - accepting connections
I20260619 20:33:31.674870 4429 mini_ranger.cc:161] Starting Ranger...
I20260619 20:33:31.690717 4429 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 20:33:32.225248 4429 mini_ranger.cc:87] Created ranger Postgres database
I20260619 20:33:32.225318 4429 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ranger-admin
2026-06-19 20:33:32,494 [I] DB FLAVOR :POSTGRES
2026-06-19 20:33:32,495 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 20:33:32,495 [I] Checking connection..
2026-06-19 20:33:32,495 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:44765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 20:33:32,737 [I] Checking connection passed.
2026-06-19 20:33:32,738 [I] --------- Verifying version history table ---------
2026-06-19 20:33:32,738 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:44765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 20:33:32,991 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 20:33:32,992 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:44765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 20:33:33,242 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 20:33:33,242 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 20:33:33,242 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:44765/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 20:33:33.494 UTC [16281] WARNING: there is no transaction in progress
2026-06-19 20:33:33,500 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 20:33:33,500 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:44765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 20:33:33,756 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 20:33:33,757 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 20:33:33,757 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:44765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 20:33:33,990 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:44765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 20:33:34,236 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:44765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-4m58.c.gcp-upstream.internal','N') ;"
2026-06-19 20:33:34,492 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:44765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 20:33:34,743 [I] Table x_portal_user does not exist in database ranger
2026-06-19 20:33:34,743 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:44765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 20:33:34,998 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 20:33:34,998 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:44765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 20:33:35,266 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:44765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 20:33:35,520 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 20:33:35,521 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:44765/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 20:33:36.513 UTC [16389] WARNING: there is no transaction in progress
2026-06-19 20:33:36.530 UTC [16389] WARNING: there is no transaction in progress
2026-06-19 20:33:36.547 UTC [16389] WARNING: there is no transaction in progress
2026-06-19 20:33:36.565 UTC [16389] WARNING: there is no transaction in progress
2026-06-19 20:33:36.582 UTC [16389] WARNING: there is no transaction in progress
2026-06-19 20:33:36.602 UTC [16389] WARNING: there is no transaction in progress
2026-06-19 20:33:36.660 UTC [16389] WARNING: there is no transaction in progress
2026-06-19 20:33:36.666 UTC [16389] WARNING: there is no transaction in progress
2026-06-19 20:33:36.682 UTC [16389] WARNING: there is no transaction in progress
2026-06-19 20:33:36.693 UTC [16389] WARNING: there is no transaction in progress
2026-06-19 20:33:36.704 UTC [16389] WARNING: there is no transaction in progress
2026-06-19 20:33:36.719 UTC [16389] WARNING: there is no transaction in progress
2026-06-19 20:33:36.729 UTC [16389] WARNING: there is no transaction in progress
2026-06-19 20:33:36.735 UTC [16389] WARNING: there is no transaction in progress
2026-06-19 20:33:36.742 UTC [16389] WARNING: there is no transaction in progress
2026-06-19 20:33:36.749 UTC [16389] WARNING: there is no transaction in progress
2026-06-19 20:33:37,577 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 20:33:37,578 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:44765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 20:33:37,847 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 20:33:37,848 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:44765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 20:33:38,126 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 20:33:38,126 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:44765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 20:33:38,411 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 20:33:38,411 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:44765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 20:33:38,688 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:44765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 20:33:38,930 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:44765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-4m58.c.gcp-upstream.internal';"
2026-06-19 20:33:39,184 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 20:33:39,184 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:44765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 20:33:39,421 [I] DB_PATCHES have already been applied
I20260619 20:33:39.426944 4429 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ranger-admin:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskPZCFKi/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 20:33:39.427016 4429 mini_ranger.cc:193] Using host: 127.4.83.84
I20260619 20:33:39.429951 4429 mini_ranger.cc:227] Ranger admin URL: http://127.4.83.84:37639
Jun 19, 2026 8:33:39 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 8:33:39 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 8:33:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 8:33:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 8:33:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 8:33:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 8:33:40 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 20:33:40.435228 4429 mini_ranger.cc:160] Time spent starting Ranger: real 8.760s user 0.000s sys 0.008s
Jun 19, 2026 8:34:01 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 8:34:01 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 19, 2026 8:34:01 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 8:34:01 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 8:34:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 8:34:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 8:34:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 8:34:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 8:34:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 8:34:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 8:34:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 8:34:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 8:34:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 8:34:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 8:34:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 8:34:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 8:34:01 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 8:34:02 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 20:34:03.454963 4429 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.4.83.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.4.83.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.4.83.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/rangerkms_127.4.83.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.4.83.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/rangerkms_127.4.83.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.83.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.4.83.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.83.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/HTTP_127.4.83.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.83.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/HTTP_127.4.83.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 20:34:03 dist-test-slave-4m58 krb5kdc[16176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901243, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 20:34:03.537781 4429 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 20:34:03.547778 4429 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 20:34:04.449896 4429 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 20:34:04.449988 4429 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ranger-kms
I20260619 20:34:04.450004 4429 mini_ranger_kms.cc:227] Using postgres at 127.4.83.84:44765
2026-06-19 20:34:04,612 [I] DB FLAVOR :POSTGRES
2026-06-19 20:34:04,613 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 20:34:04,613 [I] Checking connection
2026-06-19 20:34:04,613 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:44765/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 20:34:04,895 [I] connection success
2026-06-19 20:34:04,896 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 20:34:04,896 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 20:34:04,896 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:44765/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 20:34:05,198 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 20:34:05,198 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 20:34:05,198 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 20:34:05,198 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.83.84:44765/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 20:34:05,497 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 20:34:05.999782 4429 mini_ranger_kms.cc:310] Created kms service
I20260619 20:34:06.240525 4429 mini_ranger_kms.cc:326] Created kudu user
I20260619 20:34:06.285688 4429 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 20:34:06 dist-test-slave-4m58 krb5kdc[16176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901243, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.4.83.84@KRBTEST.COM
I20260619 20:34:06.699478 4429 mini_ranger_kms.cc:384] Added ranger policy
I20260619 20:34:06.699703 4429 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ranger-kms:/tmp/dist-test-taskPZCFKi/build/release/bin/postgresql.jar:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskPZCFKi/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 20:34:06.699723 4429 mini_ranger_kms.cc:240] Using host: 127.4.83.84
I20260619 20:34:06.703169 4429 mini_ranger_kms.cc:276] Ranger KMS PID: 16573
I20260619 20:34:06.703255 4429 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.4.83.84:58643
Jun 19, 2026 8:34:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 8:34:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 8:34:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 8:34:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 8:34:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 8:34:08 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 20:34:08.764518 4429 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.227s user 0.000s sys 0.141s
I20260619 20:34:08.764634 4429 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 20:34:08.764679 4429 mini_ranger_kms.cc:401] 127.4.83.84:58643/kms/v1/keys
Jun 19 20:34:09 dist-test-slave-4m58 krb5kdc[16176](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781901249, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.83.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 20:34:09 dist-test-slave-4m58 krb5kdc[16176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901249, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.83.84@KRBTEST.COM for HTTP/127.4.83.84@KRBTEST.COM
Jun 19 20:34:09 dist-test-slave-4m58 krb5kdc[16176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901249, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.83.84@KRBTEST.COM for HTTP/127.4.83.84@KRBTEST.COM
Jun 19 20:34:12 dist-test-slave-4m58 krb5kdc[16176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
I20260619 20:34:12.302989 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:39605
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:39915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:39605
--ranger_config_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.83.84:58643/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:34:12.383622 16623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:34:12.383909 16623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:34:12.383944 16623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:34:12.385185 16623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:34:12.385236 16623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:34:12.385254 16623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:34:12.385272 16623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:34:12.385286 16623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:34:12.386973 16623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:39915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/wal
--ranger_kms_url=127.4.83.84:58643/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:39605
--ranger_config_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:39605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:34:12.387233 16623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:34:12.387445 16623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:34:12.392676 16623 server_base.cc:1034] running on GCE node
W20260619 20:34:12.392675 16631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:12.392679 16628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:12.392976 16629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:12.393309 16623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:34:12.393642 16623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:34:12.394809 16623 hybrid_clock.cc:648] HybridClock initialized: now 1781901252394797 us; error 28 us; skew 500 ppm
Jun 19 20:34:12 dist-test-slave-4m58 krb5kdc[16176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901252, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:34:12.400997 16623 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:34:12.401551 16623 webserver.cc:460] Webserver started at http://127.4.83.126:36143/ using document root <none> and password file <none>
I20260619 20:34:12.401788 16623 fs_manager.cc:359] Metadata directory not provided
I20260619 20:34:12.401832 16623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:34:12.401968 16623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 20:34:12 dist-test-slave-4m58 krb5kdc[16176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901252, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for HTTP/127.4.83.84@KRBTEST.COM
I20260619 20:34:12.464906 16623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "bb4877a8cc1a493cb445de67acc67e42"
format_stamp: "Formatted at 2026-06-19 20:34:12 on dist-test-slave-4m58"
server_key: "4f49ff376a67023eff294b854598ab45"
server_key_iv: "627bb20e2deb856d92aef1ed63c6239d"
server_key_version: "kuduclusterkey@0"
I20260619 20:34:12.465284 16623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "bb4877a8cc1a493cb445de67acc67e42"
format_stamp: "Formatted at 2026-06-19 20:34:12 on dist-test-slave-4m58"
server_key: "4f49ff376a67023eff294b854598ab45"
server_key_iv: "627bb20e2deb856d92aef1ed63c6239d"
server_key_version: "kuduclusterkey@0"
I20260619 20:34:12.466640 16623 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:34:12.482048 16638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:12.484057 16623 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 20:34:12.484202 16623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "bb4877a8cc1a493cb445de67acc67e42"
format_stamp: "Formatted at 2026-06-19 20:34:12 on dist-test-slave-4m58"
server_key: "4f49ff376a67023eff294b854598ab45"
server_key_iv: "627bb20e2deb856d92aef1ed63c6239d"
server_key_version: "kuduclusterkey@0"
I20260619 20:34:12.484298 16623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:34:12.497541 16623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:34:12.499487 16623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:34:12.499634 16623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:34:12.504185 16623 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:39605
I20260619 20:34:12.504249 16691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:39605 every 8 connection(s)
I20260619 20:34:12.504617 16623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:34:12.505287 16692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:34:12.507614 16692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42: Bootstrap starting.
I20260619 20:34:12.508263 16692 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42: Neither blocks nor log segments found. Creating new log.
I20260619 20:34:12.508538 16692 log.cc:824] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42: Log is configured to *not* fsync() on all Append() calls
I20260619 20:34:12.509202 16692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42: No bootstrap required, opened a new log
I20260619 20:34:12.509569 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 16623
I20260619 20:34:12.509678 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:34:12.509861 4429 external_mini_cluster.cc:1428] Setting key 6563d51d404d2814d50361af6fb2816f
I20260619 20:34:12.511816 16692 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4877a8cc1a493cb445de67acc67e42" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 39605 } }
I20260619 20:34:12.512032 16692 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:34:12.512075 16692 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb4877a8cc1a493cb445de67acc67e42, State: Initialized, Role: FOLLOWER
I20260619 20:34:12.512264 16692 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4877a8cc1a493cb445de67acc67e42" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 39605 } }
I20260619 20:34:12.512346 16692 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:34:12.512387 16692 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:34:12.512450 16692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:34:12.513134 16692 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4877a8cc1a493cb445de67acc67e42" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 39605 } }
I20260619 20:34:12.513253 16692 leader_election.cc:302] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb4877a8cc1a493cb445de67acc67e42; no voters:
I20260619 20:34:12.513441 16692 leader_election.cc:288] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:34:12.513505 16697 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:34:12.513731 16697 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [term 1 LEADER]: Becoming Leader. State: Replica: bb4877a8cc1a493cb445de67acc67e42, State: Running, Role: LEADER
I20260619 20:34:12.513794 16692 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:34:12.515072 16697 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4877a8cc1a493cb445de67acc67e42" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 39605 } }
I20260619 20:34:12.515398 16697 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb4877a8cc1a493cb445de67acc67e42. Latest consensus state: current_term: 1 leader_uuid: "bb4877a8cc1a493cb445de67acc67e42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4877a8cc1a493cb445de67acc67e42" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 39605 } } }
I20260619 20:34:12.515456 16697 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [sys.catalog]: This master's current role is: LEADER
I20260619 20:34:12.515645 16692 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 20:34:12.517674 16698 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb4877a8cc1a493cb445de67acc67e42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4877a8cc1a493cb445de67acc67e42" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 39605 } } }
I20260619 20:34:12.517879 16698 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [sys.catalog]: This master's current role is: LEADER
Jun 19 20:34:12 dist-test-slave-4m58 krb5kdc[16176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:34:12.519929 16711 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:34:12.520596 16711 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:34:12.522408 16716 mvcc.cc:205] Tried to move back new op lower bound from 7298667530329939968 to 7298667530302144512. Current Snapshot: MvccSnapshot[applied={T|T < 7298667530329939968}]
I20260619 20:34:12.523113 16711 catalog_manager.cc:1347] Generated new cluster ID: dc1bf72e4dfb45eaaa4c4380f087035a
I20260619 20:34:12.523154 16711 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:34:12.523412 16696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:12.510525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60928 (local address 127.4.83.126:39605)
0619 20:34:12.510745 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:12.510755 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:12.510787 (+ 32us) server_negotiation.cc:407] Connection header received
0619 20:34:12.511790 (+ 1003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:12.511800 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:12.511942 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:12.512109 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:12.512981 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:12.514340 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:12.515290 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:12.515932 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:12.516040 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:12.518959 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:12.519006 (+ 47us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:12.519006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:12.519036 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:12.521313 (+ 2277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:12.521755 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:12.521758 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:12.521758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:12.521862 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:12.522061 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:12.522062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:12.522062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:12.522403 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:12.522917 (+ 514us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:12.522989 (+ 72us) server_negotiation.cc:299] Negotiation successful
0619 20:34:12.523153 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":88,"threads_started":1}
I20260619 20:34:12.529750 16711 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:34:12.530553 16711 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:34:12.538268 16711 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42: Generated new TSK 0
I20260619 20:34:12.538556 16711 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.83.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
I20260619 20:34:13.254473 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.83.84:58643/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:0
--local_ip_for_outbound_sockets=127.4.83.65
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:39605
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:39915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:34:13.374001 16738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:34:13.374298 16738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:34:13.374336 16738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:34:13.376179 16738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:34:13.376248 16738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:34:13.376291 16738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:34:13.378563 16738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:39915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.4.83.84:58643/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:39605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:34:13.378850 16738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:34:13.379136 16738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:34:13.385501 16743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:13.385613 16744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:13.385793 16746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:13.387048 16738 server_base.cc:1034] running on GCE node
I20260619 20:34:13.387439 16738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:34:13.387794 16738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:34:13.388996 16738 hybrid_clock.cc:648] HybridClock initialized: now 1781901253388978 us; error 38 us; skew 500 ppm
Jun 19 20:34:13 dist-test-slave-4m58 krb5kdc[16176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901253, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:34:13.391754 16738 init.cc:377] Logged in from keytab as kudu/127.4.83.65@KRBTEST.COM (short username kudu)
I20260619 20:34:13.392345 16738 webserver.cc:460] Webserver started at http://127.4.83.65:44379/ using document root <none> and password file <none>
I20260619 20:34:13.392577 16738 fs_manager.cc:359] Metadata directory not provided
I20260619 20:34:13.392627 16738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:34:13.392750 16738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 20:34:13 dist-test-slave-4m58 krb5kdc[16176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901253, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for HTTP/127.4.83.84@KRBTEST.COM
I20260619 20:34:13.408535 16738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/data/instance:
uuid: "10829b22afa1485988e14740ab83245f"
format_stamp: "Formatted at 2026-06-19 20:34:13 on dist-test-slave-4m58"
server_key: "6a03e9332b0ba18659692b2267db53e5"
server_key_iv: "4320af467c2de1e85bf27b9d156e806e"
server_key_version: "kuduclusterkey@0"
I20260619 20:34:13.408936 16738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance:
uuid: "10829b22afa1485988e14740ab83245f"
format_stamp: "Formatted at 2026-06-19 20:34:13 on dist-test-slave-4m58"
server_key: "6a03e9332b0ba18659692b2267db53e5"
server_key_iv: "4320af467c2de1e85bf27b9d156e806e"
server_key_version: "kuduclusterkey@0"
I20260619 20:34:13.411787 16738 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260619 20:34:13.426630 16753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:13.426913 16738 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:34:13.427026 16738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "10829b22afa1485988e14740ab83245f"
format_stamp: "Formatted at 2026-06-19 20:34:13 on dist-test-slave-4m58"
server_key: "6a03e9332b0ba18659692b2267db53e5"
server_key_iv: "4320af467c2de1e85bf27b9d156e806e"
server_key_version: "kuduclusterkey@0"
I20260619 20:34:13.427126 16738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:34:13.454370 16738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:34:13.456256 16738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:34:13.456369 16738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:34:13.456605 16738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:34:13.456915 16738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:34:13.456948 16738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:13.456977 16738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:34:13.457001 16738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:13.462816 16738 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:45943
I20260619 20:34:13.463328 16869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:45943 every 8 connection(s)
I20260619 20:34:13.463341 16738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
Jun 19 20:34:13 dist-test-slave-4m58 krb5kdc[16176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901253, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:34:13.473518 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 16738
I20260619 20:34:13.473623 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance
I20260619 20:34:13.473820 4429 external_mini_cluster.cc:1428] Setting key 4029c31901218bac734301084df179cf
I20260619 20:34:13.477416 16873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:13.464338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:46697 (local address 127.4.83.126:39605)
0619 20:34:13.467978 (+ 3640us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:13.467980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:13.467993 (+ 13us) server_negotiation.cc:407] Connection header received
0619 20:34:13.468021 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:13.468022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:13.468036 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:13.468133 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:13.468932 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:13.469670 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:13.471165 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:13.471363 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:13.471421 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:13.473781 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:13.473782 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:13.473783 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:13.473791 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:13.475599 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:13.476205 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:13.476206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:13.476207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:13.476282 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:13.476611 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:13.476613 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:13.476613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:13.476775 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:13.476857 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:13.477211 (+ 354us) server_negotiation.cc:299] Negotiation successful
0619 20:34:13.477313 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3600,"thread_start_us":92,"threads_started":1}
I20260619 20:34:13.477445 16872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:13.464311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:39605 (local address 127.4.83.65:46697)
0619 20:34:13.464499 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:13.464549 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:13.465215 (+ 666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:13.468257 (+ 3042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:13.468262 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:13.468522 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:34:13.468830 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:13.468836 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:13.469767 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:13.469769 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:13.471061 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:13.471064 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:13.471466 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:13.471467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:13.471613 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:13.472203 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:34:13.472213 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:34:13.473696 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:34:13.475764 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:13.475772 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:13.475775 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:13.476123 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:13.476354 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:13.476354 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:13.476355 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:13.476496 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:13.476853 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:34:13.476859 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:34:13.477046 (+ 187us) client_negotiation.cc:769] Sending connection context
0619 20:34:13.477145 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 20:34:13.477272 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":89,"threads_started":1}
I20260619 20:34:13.478171 16870 heartbeater.cc:347] Connected to a master server at 127.4.83.126:39605
I20260619 20:34:13.478297 16870 heartbeater.cc:464] Registering TS with master...
I20260619 20:34:13.478513 16870 heartbeater.cc:511] Master 127.4.83.126:39605 requested a full tablet report, sending...
I20260619 20:34:13.479146 16655 ts_manager.cc:194] Registered new tserver with Master: 10829b22afa1485988e14740ab83245f (127.4.83.65:45943)
I20260619 20:34:13.480221 16655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.65@KRBTEST.COM'} at 127.4.83.65:46697
WARNING: no policy specified for kudu/127.4.83.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
I20260619 20:34:13.502821 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.83.84:58643/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:0
--local_ip_for_outbound_sockets=127.4.83.66
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:39605
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:39915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:34:13.622603 16876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:34:13.622897 16876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:34:13.622941 16876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:34:13.624801 16876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:34:13.624915 16876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:34:13.624976 16876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:34:13.627470 16876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:39915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.4.83.84:58643/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:39605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:34:13.627956 16876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:34:13.628289 16876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:34:13.636029 16883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:13.636312 16886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:13.636309 16876 server_base.cc:1034] running on GCE node
Jun 19 20:34:13 dist-test-slave-4m58 krb5kdc[16176](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781901253, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260619 20:34:13.636039 16884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:13.637130 16876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:34:13.637454 16876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:34:13.638664 16876 hybrid_clock.cc:648] HybridClock initialized: now 1781901253638639 us; error 35 us; skew 500 ppm
Jun 19 20:34:13 dist-test-slave-4m58 krb5kdc[16176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901253, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:34:13.642055 16876 init.cc:377] Logged in from keytab as kudu/127.4.83.66@KRBTEST.COM (short username kudu)
I20260619 20:34:13.642616 16876 webserver.cc:460] Webserver started at http://127.4.83.66:35251/ using document root <none> and password file <none>
I20260619 20:34:13.642936 16876 fs_manager.cc:359] Metadata directory not provided
I20260619 20:34:13.643002 16876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:34:13.643177 16876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 20:34:13 dist-test-slave-4m58 krb5kdc[16176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901253, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for HTTP/127.4.83.84@KRBTEST.COM
I20260619 20:34:13.661226 16876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/data/instance:
uuid: "b1f73e27c48c475fa6240020b1174219"
format_stamp: "Formatted at 2026-06-19 20:34:13 on dist-test-slave-4m58"
server_key: "2c9d5f899693702754d0b4d159a9b566"
server_key_iv: "5ba8947f18218a56cf5d52d1e77d7ca0"
server_key_version: "kuduclusterkey@0"
I20260619 20:34:13.662038 16876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance:
uuid: "b1f73e27c48c475fa6240020b1174219"
format_stamp: "Formatted at 2026-06-19 20:34:13 on dist-test-slave-4m58"
server_key: "2c9d5f899693702754d0b4d159a9b566"
server_key_iv: "5ba8947f18218a56cf5d52d1e77d7ca0"
server_key_version: "kuduclusterkey@0"
I20260619 20:34:13.664474 16876 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260619 20:34:13.684358 16893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:13.684671 16876 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 20:34:13.684777 16876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "b1f73e27c48c475fa6240020b1174219"
format_stamp: "Formatted at 2026-06-19 20:34:13 on dist-test-slave-4m58"
server_key: "2c9d5f899693702754d0b4d159a9b566"
server_key_iv: "5ba8947f18218a56cf5d52d1e77d7ca0"
server_key_version: "kuduclusterkey@0"
I20260619 20:34:13.685173 16876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:34:13.697860 16876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:34:13.699985 16876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:34:13.700115 16876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:34:13.700331 16876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:34:13.700693 16876 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:34:13.700999 16876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:13.701072 16876 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:34:13.701164 16876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260619 20:34:13.707574 16876 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:44779
I20260619 20:34:13.708029 16876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
I20260619 20:34:13.710551 17007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:44779 every 8 connection(s)
I20260619 20:34:13.712379 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 16876
I20260619 20:34:13.712486 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance
I20260619 20:34:13.712729 4429 external_mini_cluster.cc:1428] Setting key 06b775a3bcb95a0d7efa9efb73839f4c
Jun 19 20:34:13 dist-test-slave-4m58 krb5kdc[16176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901253, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.83.67@KRBTEST.COM; defaulting to no policy
I20260619 20:34:13.723600 16873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:13.708790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:49385 (local address 127.4.83.126:39605)
0619 20:34:13.708848 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:13.708850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:13.709015 (+ 165us) server_negotiation.cc:407] Connection header received
0619 20:34:13.709597 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:13.709599 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:13.709611 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:13.709695 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:13.710252 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:13.710973 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:13.712399 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:13.712717 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:13.712762 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:13.715738 (+ 2976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:13.715740 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:13.715741 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:13.715749 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:13.717366 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:13.717888 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:13.717889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:13.717889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:13.717951 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:13.718235 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:13.718236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:13.718236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:13.718386 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:13.718427 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:13.723320 (+ 4893us) server_negotiation.cc:299] Negotiation successful
0619 20:34:13.723447 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 20:34:13.724242 17010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:13.708733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:39605 (local address 127.4.83.66:49385)
0619 20:34:13.708884 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:13.708925 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:13.709473 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:13.709753 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:13.709757 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:13.709953 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:34:13.710154 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:13.710158 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:13.711082 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:13.711083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:13.712282 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:13.712286 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:13.712873 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:13.712874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:13.713133 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:13.713752 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:34:13.713763 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:34:13.715624 (+ 1861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:34:13.717465 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:13.717473 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:13.717476 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:13.717802 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:13.718022 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:13.718022 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:13.718023 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:13.718162 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:13.718474 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:34:13.718481 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:34:13.723083 (+ 4602us) client_negotiation.cc:769] Sending connection context
0619 20:34:13.723752 (+ 669us) client_negotiation.cc:240] Negotiation successful
0619 20:34:13.723979 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
I20260619 20:34:13.724880 17008 heartbeater.cc:347] Connected to a master server at 127.4.83.126:39605
I20260619 20:34:13.724992 17008 heartbeater.cc:464] Registering TS with master...
I20260619 20:34:13.725208 17008 heartbeater.cc:511] Master 127.4.83.126:39605 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.67@KRBTEST.COM" created.
I20260619 20:34:13.725657 16655 ts_manager.cc:194] Registered new tserver with Master: b1f73e27c48c475fa6240020b1174219 (127.4.83.66:44779)
I20260619 20:34:13.726161 16655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.66@KRBTEST.COM'} at 127.4.83.66:49385
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
I20260619 20:34:13.747179 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.83.84:58643/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:0
--local_ip_for_outbound_sockets=127.4.83.67
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:39605
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:39915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:34:13.864638 17014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:34:13.864941 17014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:34:13.864981 17014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:34:13.866850 17014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:34:13.866922 17014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:34:13.866966 17014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:34:13.869380 17014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:39915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.4.83.84:58643/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:39605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:34:13.869652 17014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:34:13.869987 17014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:34:13.876297 17019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:13.876289 17022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:13.876289 17020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:13.876699 17014 server_base.cc:1034] running on GCE node
I20260619 20:34:13.876901 17014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:34:13.877161 17014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:34:13.878310 17014 hybrid_clock.cc:648] HybridClock initialized: now 1781901253878300 us; error 28 us; skew 500 ppm
Jun 19 20:34:13 dist-test-slave-4m58 krb5kdc[16176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901253, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:34:13.880712 17014 init.cc:377] Logged in from keytab as kudu/127.4.83.67@KRBTEST.COM (short username kudu)
I20260619 20:34:13.881175 17014 webserver.cc:460] Webserver started at http://127.4.83.67:38293/ using document root <none> and password file <none>
I20260619 20:34:13.881395 17014 fs_manager.cc:359] Metadata directory not provided
I20260619 20:34:13.881438 17014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:34:13.881557 17014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 20:34:13 dist-test-slave-4m58 krb5kdc[16176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901253, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for HTTP/127.4.83.84@KRBTEST.COM
I20260619 20:34:13.893900 17014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/data/instance:
uuid: "770401e91720425c8821923b5db0f43e"
format_stamp: "Formatted at 2026-06-19 20:34:13 on dist-test-slave-4m58"
server_key: "228240ea160db3f001f64b0fa9ce7adc"
server_key_iv: "65dbd267d2e6bac9aaabb8dc3713dc53"
server_key_version: "kuduclusterkey@0"
I20260619 20:34:13.894361 17014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance:
uuid: "770401e91720425c8821923b5db0f43e"
format_stamp: "Formatted at 2026-06-19 20:34:13 on dist-test-slave-4m58"
server_key: "228240ea160db3f001f64b0fa9ce7adc"
server_key_iv: "65dbd267d2e6bac9aaabb8dc3713dc53"
server_key_version: "kuduclusterkey@0"
I20260619 20:34:13.898480 17014 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.001s
I20260619 20:34:13.928232 17029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:13.939802 17014 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.000s sys 0.001s
I20260619 20:34:13.939975 17014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "770401e91720425c8821923b5db0f43e"
format_stamp: "Formatted at 2026-06-19 20:34:13 on dist-test-slave-4m58"
server_key: "228240ea160db3f001f64b0fa9ce7adc"
server_key_iv: "65dbd267d2e6bac9aaabb8dc3713dc53"
server_key_version: "kuduclusterkey@0"
I20260619 20:34:13.940083 17014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:34:13.961625 17014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:34:13.963692 17014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:34:13.963910 17014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:34:13.964170 17014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:34:13.964505 17014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:34:13.964541 17014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:13.964571 17014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:34:13.964618 17014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:13.970945 17014 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:45943
I20260619 20:34:13.971175 17142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:45943 every 8 connection(s)
I20260619 20:34:13.971433 17014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
I20260619 20:34:13.977674 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 17014
I20260619 20:34:13.977797 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance
I20260619 20:34:13.977972 4429 external_mini_cluster.cc:1428] Setting key 08a86ac03c2799da2bdc612583e450f6
Jun 19 20:34:13 dist-test-slave-4m58 krb5kdc[16176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901253, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:34:13.985263 16873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:13.972435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:40067 (local address 127.4.83.126:39605)
0619 20:34:13.972494 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:13.972497 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:13.976415 (+ 3918us) server_negotiation.cc:407] Connection header received
0619 20:34:13.976955 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:13.976957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:13.976971 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:13.977023 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:13.977681 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:13.978397 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:13.979505 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:13.979702 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:13.979757 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:13.982500 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:13.982502 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:13.982502 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:13.982510 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:13.983773 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:13.984203 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:13.984204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:13.984205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:13.984258 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:13.984488 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:13.984489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:13.984489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:13.984609 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:13.984662 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:13.985063 (+ 401us) server_negotiation.cc:299] Negotiation successful
0619 20:34:13.985154 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 20:34:13.985569 17145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:13.972159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:39605 (local address 127.4.83.67:40067)
0619 20:34:13.976254 (+ 4095us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:13.976301 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:13.976850 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:13.977100 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:13.977103 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:13.977308 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:34:13.977567 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:13.977573 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:13.978511 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:13.978512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:13.979427 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:13.979430 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:13.979791 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:13.979792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:13.979960 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:13.980485 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:34:13.980496 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:34:13.982412 (+ 1916us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:34:13.983888 (+ 1476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:13.983896 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:13.983899 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:13.984138 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:13.984311 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:13.984311 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:13.984312 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:13.984428 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:13.984663 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:34:13.984669 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:34:13.984828 (+ 159us) client_negotiation.cc:769] Sending connection context
0619 20:34:13.984997 (+ 169us) client_negotiation.cc:240] Negotiation successful
0619 20:34:13.985180 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4050,"thread_start_us":73,"threads_started":1}
I20260619 20:34:13.986018 17143 heartbeater.cc:347] Connected to a master server at 127.4.83.126:39605
I20260619 20:34:13.986123 17143 heartbeater.cc:464] Registering TS with master...
I20260619 20:34:13.986299 17143 heartbeater.cc:511] Master 127.4.83.126:39605 requested a full tablet report, sending...
I20260619 20:34:13.986656 16655 ts_manager.cc:194] Registered new tserver with Master: 770401e91720425c8821923b5db0f43e (127.4.83.67:45943)
I20260619 20:34:13.987071 16655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.67@KRBTEST.COM'} at 127.4.83.67:40067
I20260619 20:34:13.989589 4429 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:34:13.990302 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 16738
I20260619 20:34:13.996449 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 16876
I20260619 20:34:14.002640 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 17014
I20260619 20:34:14.007117 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 16623
I20260619 20:34:14.011910 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:39605
--webserver_interface=127.4.83.126
--webserver_port=36143
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:39915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:39605
--ranger_config_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.83.84:58643/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.83.84:58643/kms with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:34:14.097517 17146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:34:14.097755 17146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:34:14.097788 17146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:34:14.099011 17146 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:34:14.099056 17146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:34:14.099076 17146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:34:14.099092 17146 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:34:14.099107 17146 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:34:14.100780 17146 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:39915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.83.84:58643/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:39605
--ranger_config_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:39605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=36143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:34:14.101037 17146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:34:14.101250 17146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:34:14.106122 17152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:14.106182 17151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:14.106212 17146 server_base.cc:1034] running on GCE node
W20260619 20:34:14.106084 17154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:14.106516 17146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:34:14.106774 17146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:34:14.107906 17146 hybrid_clock.cc:648] HybridClock initialized: now 1781901254107892 us; error 41 us; skew 500 ppm
Jun 19 20:34:14 dist-test-slave-4m58 krb5kdc[16176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901254, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:34:14.110713 17146 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:34:14.111213 17146 webserver.cc:460] Webserver started at http://127.4.83.126:36143/ using document root <none> and password file <none>
I20260619 20:34:14.111431 17146 fs_manager.cc:359] Metadata directory not provided
I20260619 20:34:14.111491 17146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 20:34:14 dist-test-slave-4m58 krb5kdc[16176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901254, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for HTTP/127.4.83.84@KRBTEST.COM
I20260619 20:34:14.153100 17146 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:34:14.153820 17162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:14.154017 17146 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:34:14.154119 17146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "bb4877a8cc1a493cb445de67acc67e42"
format_stamp: "Formatted at 2026-06-19 20:34:12 on dist-test-slave-4m58"
server_key: "4f49ff376a67023eff294b854598ab45"
server_key_iv: "627bb20e2deb856d92aef1ed63c6239d"
server_key_version: "kuduclusterkey@0"
I20260619 20:34:14.154487 17146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:34:14.169629 17146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:34:14.171257 17146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:34:14.171401 17146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:34:14.175607 17146 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:39605
I20260619 20:34:14.175645 17216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:39605 every 8 connection(s)
I20260619 20:34:14.176051 17146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:34:14.177297 17217 sys_catalog.cc:263] Verifying existing consensus state
I20260619 20:34:14.177693 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 17146
I20260619 20:34:14.177980 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.83.84:58643/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:45943
--local_ip_for_outbound_sockets=127.4.83.65
--tserver_master_addrs=127.4.83.126:39605
--webserver_port=44379
--webserver_interface=127.4.83.65
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:39915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.83.84:58643/kms with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:34:14.178117 17217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42: Bootstrap starting.
I20260619 20:34:14.186448 17217 log.cc:824] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42: Log is configured to *not* fsync() on all Append() calls
I20260619 20:34:14.188333 17217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:34:14.188606 17217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42: Bootstrap complete.
I20260619 20:34:14.190781 17217 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4877a8cc1a493cb445de67acc67e42" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 39605 } }
I20260619 20:34:14.191015 17217 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb4877a8cc1a493cb445de67acc67e42, State: Initialized, Role: FOLLOWER
I20260619 20:34:14.191167 17217 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4877a8cc1a493cb445de67acc67e42" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 39605 } }
I20260619 20:34:14.191231 17217 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:34:14.191269 17217 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:34:14.191313 17217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [term 1 FOLLOWER]: Advancing to term 2
I20260619 20:34:14.192436 17217 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4877a8cc1a493cb445de67acc67e42" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 39605 } }
I20260619 20:34:14.192589 17217 leader_election.cc:302] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb4877a8cc1a493cb445de67acc67e42; no voters:
I20260619 20:34:14.192756 17217 leader_election.cc:288] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 20:34:14.192803 17221 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 20:34:14.192947 17221 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [term 2 LEADER]: Becoming Leader. State: Replica: bb4877a8cc1a493cb445de67acc67e42, State: Running, Role: LEADER
I20260619 20:34:14.193035 17217 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:34:14.193053 17221 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4877a8cc1a493cb445de67acc67e42" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 39605 } }
I20260619 20:34:14.193351 17221 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb4877a8cc1a493cb445de67acc67e42. Latest consensus state: current_term: 2 leader_uuid: "bb4877a8cc1a493cb445de67acc67e42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4877a8cc1a493cb445de67acc67e42" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 39605 } } }
I20260619 20:34:14.193563 17221 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [sys.catalog]: This master's current role is: LEADER
I20260619 20:34:14.193498 17222 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bb4877a8cc1a493cb445de67acc67e42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4877a8cc1a493cb445de67acc67e42" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 39605 } } }
I20260619 20:34:14.193754 17222 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42 [sys.catalog]: This master's current role is: LEADER
I20260619 20:34:14.194007 17230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:34:14.194496 17230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:34:14.194676 17230 catalog_manager.cc:1259] Loaded cluster ID: dc1bf72e4dfb45eaaa4c4380f087035a
I20260619 20:34:14.194754 17230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:34:14.194859 17217 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 20:34:14.198060 17230 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:34:14.198642 17230 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42: Loaded TSK: 0
I20260619 20:34:14.198938 17230 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 20:34:14.292924 17219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:34:14.293253 17219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:34:14.293291 17219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:34:14.295231 17219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:34:14.295378 17219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:34:14.295466 17219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:34:14.297845 17219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:39915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.83.84:58643/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:45943
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=44379
--tserver_master_addrs=127.4.83.126:39605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:34:14.298484 17219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:34:14.298844 17219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:34:14.305897 17257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:14.306313 17254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:14.306285 17219 server_base.cc:1034] running on GCE node
W20260619 20:34:14.305959 17255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:14.306761 17219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:34:14.307076 17219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:34:14.308243 17219 hybrid_clock.cc:648] HybridClock initialized: now 1781901254308217 us; error 42 us; skew 500 ppm
Jun 19 20:34:14 dist-test-slave-4m58 krb5kdc[16176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901254, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:34:14.311594 17219 init.cc:377] Logged in from keytab as kudu/127.4.83.65@KRBTEST.COM (short username kudu)
I20260619 20:34:14.312238 17219 webserver.cc:460] Webserver started at http://127.4.83.65:44379/ using document root <none> and password file <none>
I20260619 20:34:14.312518 17219 fs_manager.cc:359] Metadata directory not provided
I20260619 20:34:14.312602 17219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 20:34:14 dist-test-slave-4m58 krb5kdc[16176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901254, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for HTTP/127.4.83.84@KRBTEST.COM
I20260619 20:34:14.337419 17219 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.000s
I20260619 20:34:14.338503 17265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:14.338660 17219 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:34:14.338740 17219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "10829b22afa1485988e14740ab83245f"
format_stamp: "Formatted at 2026-06-19 20:34:13 on dist-test-slave-4m58"
server_key: "6a03e9332b0ba18659692b2267db53e5"
server_key_iv: "4320af467c2de1e85bf27b9d156e806e"
server_key_version: "kuduclusterkey@0"
I20260619 20:34:14.339228 17219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:34:14.358783 17219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:34:14.360975 17219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:34:14.361129 17219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:34:14.361384 17219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:34:14.361737 17219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:34:14.361773 17219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:14.361804 17219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:34:14.361840 17219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:14.367807 17219 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:45943
I20260619 20:34:14.367919 17380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:45943 every 8 connection(s)
I20260619 20:34:14.368330 17219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
I20260619 20:34:14.374437 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 17219
I20260619 20:34:14.374778 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.83.84:58643/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:44779
--local_ip_for_outbound_sockets=127.4.83.66
--tserver_master_addrs=127.4.83.126:39605
--webserver_port=35251
--webserver_interface=127.4.83.66
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:39915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.83.84:58643/kms with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 20:34:14 dist-test-slave-4m58 krb5kdc[16176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901254, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:34:14.380344 17383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:14.369084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:39605 (local address 127.4.83.65:34483)
0619 20:34:14.369250 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:14.369293 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:14.369808 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:14.370234 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:14.370238 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:14.370440 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:34:14.370634 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:14.370639 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:14.371750 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:14.371751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:14.372690 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:14.372693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:14.373349 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:14.373350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:14.373473 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:14.374105 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:34:14.374114 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:34:14.375999 (+ 1885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:34:14.378155 (+ 2156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:14.378166 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:14.378168 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:14.378533 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:14.378901 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:14.378902 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:14.378903 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:14.379095 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:14.379511 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:34:14.379521 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:34:14.379706 (+ 185us) client_negotiation.cc:769] Sending connection context
0619 20:34:14.379865 (+ 159us) client_negotiation.cc:240] Negotiation successful
0619 20:34:14.380067 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":80,"threads_started":1}
I20260619 20:34:14.380380 17384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:14.369210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:34483 (local address 127.4.83.126:39605)
0619 20:34:14.369410 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:14.369418 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:14.369457 (+ 39us) server_negotiation.cc:407] Connection header received
0619 20:34:14.370024 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:14.370029 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:14.370073 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:14.370184 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:14.370747 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:14.371645 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:14.372776 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:14.373243 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:14.373343 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:14.376144 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:14.376149 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:14.376150 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:14.376162 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:14.378043 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:14.378705 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:14.378710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:14.378710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:14.378822 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:14.379167 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:14.379167 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:14.379168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:14.379432 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:14.379499 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:14.379947 (+ 448us) server_negotiation.cc:299] Negotiation successful
0619 20:34:14.380100 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":99,"threads_started":1}
I20260619 20:34:14.381712 17381 heartbeater.cc:347] Connected to a master server at 127.4.83.126:39605
I20260619 20:34:14.381821 17381 heartbeater.cc:464] Registering TS with master...
I20260619 20:34:14.382045 17381 heartbeater.cc:511] Master 127.4.83.126:39605 requested a full tablet report, sending...
W20260619 20:34:14.383019 17381 heartbeater.cc:644] Failed to heartbeat to 127.4.83.126:39605 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 20:34:14.383093 17381 heartbeater.cc:464] Registering TS with master...
I20260619 20:34:14.383175 17381 heartbeater.cc:511] Master 127.4.83.126:39605 requested a full tablet report, sending...
I20260619 20:34:14.383816 17381 heartbeater.cc:464] Registering TS with master...
I20260619 20:34:14.384034 17381 heartbeater.cc:511] Master 127.4.83.126:39605 requested a full tablet report, sending...
W20260619 20:34:14.384488 17381 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 20:34:14.495450 17385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:34:14.495888 17385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:34:14.495991 17385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:34:14.497907 17385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:34:14.498018 17385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:34:14.498111 17385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:34:14.500597 17385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:39915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.83.84:58643/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:44779
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=35251
--tserver_master_addrs=127.4.83.126:39605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:34:14.500864 17385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:34:14.501159 17385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:34:14.506642 17390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:14.506879 17391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:14.506642 17393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:14.506949 17385 server_base.cc:1034] running on GCE node
I20260619 20:34:14.507169 17385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:34:14.507483 17385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:34:14.508671 17385 hybrid_clock.cc:648] HybridClock initialized: now 1781901254508644 us; error 42 us; skew 500 ppm
Jun 19 20:34:14 dist-test-slave-4m58 krb5kdc[16176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901254, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:34:14.512063 17385 init.cc:377] Logged in from keytab as kudu/127.4.83.66@KRBTEST.COM (short username kudu)
I20260619 20:34:14.512661 17385 webserver.cc:460] Webserver started at http://127.4.83.66:35251/ using document root <none> and password file <none>
I20260619 20:34:14.512916 17385 fs_manager.cc:359] Metadata directory not provided
I20260619 20:34:14.513010 17385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 20:34:14 dist-test-slave-4m58 krb5kdc[16176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901254, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for HTTP/127.4.83.84@KRBTEST.COM
I20260619 20:34:14.532039 17385 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.000s
I20260619 20:34:14.533249 17401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:14.533524 17385 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:34:14.533628 17385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "b1f73e27c48c475fa6240020b1174219"
format_stamp: "Formatted at 2026-06-19 20:34:13 on dist-test-slave-4m58"
server_key: "2c9d5f899693702754d0b4d159a9b566"
server_key_iv: "5ba8947f18218a56cf5d52d1e77d7ca0"
server_key_version: "kuduclusterkey@0"
I20260619 20:34:14.534112 17385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:34:14.558935 17385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:34:14.560956 17385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:34:14.561094 17385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:34:14.561362 17385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:34:14.561662 17385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:34:14.561700 17385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:14.561729 17385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:34:14.561753 17385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:14.568439 17385 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:44779
I20260619 20:34:14.568518 17515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:44779 every 8 connection(s)
I20260619 20:34:14.568881 17385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
I20260619 20:34:14.575657 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 17385
Jun 19 20:34:14 dist-test-slave-4m58 krb5kdc[16176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901254, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:34:14.575999 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.83.84:58643/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:45943
--local_ip_for_outbound_sockets=127.4.83.67
--tserver_master_addrs=127.4.83.126:39605
--webserver_port=38293
--webserver_interface=127.4.83.67
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:39915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.83.84:58643/kms with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:34:14.580231 17518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:14.569587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:39605 (local address 127.4.83.66:52849)
0619 20:34:14.569747 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:14.569789 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:14.570417 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:14.570714 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:14.570718 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:14.570906 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:34:14.571162 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:14.571180 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:14.572181 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:14.572182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:14.573463 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:14.573466 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:14.573982 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:14.573983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:14.574124 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:14.574783 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:34:14.574793 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:34:14.576067 (+ 1274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:34:14.578571 (+ 2504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:14.578579 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:14.578583 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:14.578862 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:14.579066 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:14.579067 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:14.579067 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:14.579207 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:14.579477 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:34:14.579483 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:34:14.579656 (+ 173us) client_negotiation.cc:769] Sending connection context
0619 20:34:14.579767 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 20:34:14.579966 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":80,"threads_started":1}
I20260619 20:34:14.580963 17384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:14.569582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:52849 (local address 127.4.83.126:39605)
0619 20:34:14.569645 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:14.569647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:14.569890 (+ 243us) server_negotiation.cc:407] Connection header received
0619 20:34:14.570543 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:14.570545 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:14.570562 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:14.570640 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:14.571278 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:14.572072 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:14.573565 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:14.573870 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:14.573952 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:14.576157 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:14.576158 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:14.576159 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:14.576167 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:14.578496 (+ 2329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:14.578944 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:14.578945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:14.578946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:14.579006 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:14.579267 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:14.579268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:14.579268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:14.579403 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:14.579454 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:14.580774 (+ 1320us) server_negotiation.cc:299] Negotiation successful
0619 20:34:14.580868 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 20:34:14.581502 17516 heartbeater.cc:347] Connected to a master server at 127.4.83.126:39605
I20260619 20:34:14.581600 17516 heartbeater.cc:464] Registering TS with master...
I20260619 20:34:14.581804 17516 heartbeater.cc:511] Master 127.4.83.126:39605 requested a full tablet report, sending...
W20260619 20:34:14.582468 17516 heartbeater.cc:644] Failed to heartbeat to 127.4.83.126:39605 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 20:34:14.582535 17516 heartbeater.cc:464] Registering TS with master...
I20260619 20:34:14.582595 17516 heartbeater.cc:511] Master 127.4.83.126:39605 requested a full tablet report, sending...
I20260619 20:34:14.582958 17516 heartbeater.cc:464] Registering TS with master...
I20260619 20:34:14.583036 17516 heartbeater.cc:511] Master 127.4.83.126:39605 requested a full tablet report, sending...
W20260619 20:34:14.583374 17516 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 20:34:14.675393 17519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:34:14.675800 17519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:34:14.675917 17519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:34:14.677682 17519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:34:14.677744 17519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:34:14.677788 17519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:34:14.679560 17519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:39915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.83.84:58643/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:45943
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=38293
--tserver_master_addrs=127.4.83.126:39605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:34:14.679824 17519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:34:14.680146 17519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:34:14.685184 17525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:14.685225 17527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:14.685221 17524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:14.685609 17519 server_base.cc:1034] running on GCE node
I20260619 20:34:14.685810 17519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:34:14.686041 17519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:34:14.687564 17519 hybrid_clock.cc:648] HybridClock initialized: now 1781901254687537 us; error 42 us; skew 500 ppm
Jun 19 20:34:14 dist-test-slave-4m58 krb5kdc[16176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901254, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:34:14.691123 17519 init.cc:377] Logged in from keytab as kudu/127.4.83.67@KRBTEST.COM (short username kudu)
I20260619 20:34:14.691812 17519 webserver.cc:460] Webserver started at http://127.4.83.67:38293/ using document root <none> and password file <none>
I20260619 20:34:14.692164 17519 fs_manager.cc:359] Metadata directory not provided
I20260619 20:34:14.692309 17519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 20:34:14 dist-test-slave-4m58 krb5kdc[16176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901254, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for HTTP/127.4.83.84@KRBTEST.COM
I20260619 20:34:14.715653 17519 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:34:14.716717 17536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:14.716954 17519 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:34:14.717072 17519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "770401e91720425c8821923b5db0f43e"
format_stamp: "Formatted at 2026-06-19 20:34:13 on dist-test-slave-4m58"
server_key: "228240ea160db3f001f64b0fa9ce7adc"
server_key_iv: "65dbd267d2e6bac9aaabb8dc3713dc53"
server_key_version: "kuduclusterkey@0"
I20260619 20:34:14.717546 17519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:34:14.735186 17519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:34:14.737655 17519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:34:14.737810 17519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:34:14.738081 17519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:34:14.738513 17519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:34:14.738564 17519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:14.738678 17519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:34:14.738713 17519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:14.746141 17519 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:45943
I20260619 20:34:14.746655 17519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
I20260619 20:34:14.747141 17650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:45943 every 8 connection(s)
I20260619 20:34:14.752661 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 17519
Jun 19 20:34:14 dist-test-slave-4m58 krb5kdc[16176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901254, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:34:14.758606 17384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:14.747720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:36365 (local address 127.4.83.126:39605)
0619 20:34:14.747762 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:14.747764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:14.747775 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:34:14.748334 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:14.748336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:14.748348 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:14.748375 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:14.749155 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:14.749936 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:14.751371 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:14.751696 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:14.751779 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:14.755151 (+ 3372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:14.755152 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:14.755153 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:14.755161 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:14.756885 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:14.757448 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:14.757449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:14.757450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:14.757510 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:14.757782 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:14.757782 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:14.757783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:14.757921 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:14.757963 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:14.758411 (+ 448us) server_negotiation.cc:299] Negotiation successful
0619 20:34:14.758502 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:34:14.759373 17653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:14.747403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:39605 (local address 127.4.83.67:36365)
0619 20:34:14.747553 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:14.747601 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:14.748215 (+ 614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:14.748547 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:14.748551 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:14.748779 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:34:14.749037 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:14.749041 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:14.750038 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:14.750039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:14.751265 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:14.751268 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:14.751820 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:14.751821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:14.752008 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:14.753064 (+ 1056us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:34:14.753075 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:34:14.755027 (+ 1952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:34:14.757011 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:14.757021 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:14.757024 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:14.757359 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:14.757576 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:14.757577 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:14.757577 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:14.757715 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:14.758006 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:34:14.758012 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:34:14.758248 (+ 236us) client_negotiation.cc:769] Sending connection context
0619 20:34:14.758953 (+ 705us) client_negotiation.cc:240] Negotiation successful
0619 20:34:14.759139 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260619 20:34:14.760061 17651 heartbeater.cc:347] Connected to a master server at 127.4.83.126:39605
I20260619 20:34:14.760155 17651 heartbeater.cc:464] Registering TS with master...
I20260619 20:34:14.760353 17651 heartbeater.cc:511] Master 127.4.83.126:39605 requested a full tablet report, sending...
W20260619 20:34:14.761063 17651 heartbeater.cc:644] Failed to heartbeat to 127.4.83.126:39605 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 20:34:14.761139 17651 heartbeater.cc:464] Registering TS with master...
I20260619 20:34:14.761200 17651 heartbeater.cc:511] Master 127.4.83.126:39605 requested a full tablet report, sending...
I20260619 20:34:14.761551 17651 heartbeater.cc:464] Registering TS with master...
I20260619 20:34:14.761620 17651 heartbeater.cc:511] Master 127.4.83.126:39605 requested a full tablet report, sending...
W20260619 20:34:14.761922 17651 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 20:34:14.769160 17655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:14.753650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60938 (local address 127.4.83.126:39605)
0619 20:34:14.753803 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:14.753805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:14.756066 (+ 2261us) server_negotiation.cc:407] Connection header received
0619 20:34:14.762099 (+ 6033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:14.762100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:14.762118 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:14.762192 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:14.763862 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:14.764364 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:14.765101 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:14.765294 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:14.765350 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:14.766647 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:14.766649 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:14.766649 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:14.766657 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:14.768127 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:14.768428 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:14.768429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:14.768429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:14.768482 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:14.768650 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:14.768651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:14.768651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:14.768755 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:14.768937 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:14.768972 (+ 35us) server_negotiation.cc:299] Negotiation successful
0619 20:34:14.769058 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":82,"threads_started":1}
Jun 19 20:34:15 dist-test-slave-4m58 krb5kdc[16176](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781901255, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260619 20:34:15.369220 17376 messenger.cc:188] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.4.83.65:45943
I20260619 20:34:15.384727 17381 heartbeater.cc:464] Registering TS with master...
I20260619 20:34:15.384912 17381 heartbeater.cc:511] Master 127.4.83.126:39605 requested a full tablet report, sending...
I20260619 20:34:15.385357 17180 ts_manager.cc:194] Registered new tserver with Master: 10829b22afa1485988e14740ab83245f (127.4.83.65:45943)
I20260619 20:34:15.386184 17180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.65@KRBTEST.COM'} at 127.4.83.65:34483
I20260619 20:34:15.583593 17516 heartbeater.cc:464] Registering TS with master...
I20260619 20:34:15.583767 17516 heartbeater.cc:511] Master 127.4.83.126:39605 requested a full tablet report, sending...
I20260619 20:34:15.584123 17180 ts_manager.cc:194] Registered new tserver with Master: b1f73e27c48c475fa6240020b1174219 (127.4.83.66:44779)
I20260619 20:34:15.584515 17180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.66@KRBTEST.COM'} at 127.4.83.66:52849
Jun 19 20:34:15 dist-test-slave-4m58 krb5kdc[16176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901255, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for HTTP/127.4.83.84@KRBTEST.COM
W20260619 20:34:15.747476 17646 messenger.cc:188] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.4.83.67:45943
I20260619 20:34:15.762111 17651 heartbeater.cc:464] Registering TS with master...
I20260619 20:34:15.762277 17651 heartbeater.cc:511] Master 127.4.83.126:39605 requested a full tablet report, sending...
I20260619 20:34:15.762555 17180 ts_manager.cc:194] Registered new tserver with Master: 770401e91720425c8821923b5db0f43e (127.4.83.67:45943)
I20260619 20:34:15.763034 17180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.67@KRBTEST.COM'} at 127.4.83.67:36365
I20260619 20:34:15.769608 4429 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:34:15.779704 17672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:15.771576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60942 (local address 127.4.83.126:39605)
0619 20:34:15.772035 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:15.772037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:15.772043 (+ 6us) server_negotiation.cc:407] Connection header received
0619 20:34:15.772065 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:15.772065 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:15.772075 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:15.772806 (+ 731us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:15.772845 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:15.773661 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:15.774721 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:15.775014 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:15.775378 (+ 364us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:15.776577 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:15.776578 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:15.776578 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:15.776586 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:15.778215 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:15.778586 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:15.778587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:15.778587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:15.778653 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:15.778837 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:15.778837 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:15.778837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:15.778965 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:15.779461 (+ 496us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:15.779503 (+ 42us) server_negotiation.cc:299] Negotiation successful
0619 20:34:15.779584 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":88,"threads_started":1}
I20260619 20:34:15.781841 17180 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60942:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:34:15.783536 17180 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:34:15.801021 17679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:15.790074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:45943 (local address 127.0.0.1:53080)
0619 20:34:15.791194 (+ 1120us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:15.791202 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:15.791233 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:15.791796 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:15.791799 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:15.791801 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:34:15.791911 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:15.791913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:15.795501 (+ 3588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:15.795502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:15.797019 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:15.797023 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:15.800685 (+ 3662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:15.800687 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:15.800794 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:15.800813 (+ 19us) client_negotiation.cc:769] Sending connection context
0619 20:34:15.800862 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 20:34:15.800919 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1105,"mutex_wait_us":2,"thread_start_us":84,"threads_started":1}
I20260619 20:34:15.802076 17682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:15.790376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53080 (local address 127.4.83.67:45943)
0619 20:34:15.791452 (+ 1076us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:15.791457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:15.791466 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:34:15.791500 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:15.791503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:15.791518 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:15.791985 (+ 467us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:34:15.792051 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:15.795428 (+ 3377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:15.797135 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:15.800583 (+ 3448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:15.801821 (+ 1238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:15.801910 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:15.801937 (+ 27us) server_negotiation.cc:299] Negotiation successful
0619 20:34:15.801989 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1038,"thread_start_us":54,"threads_started":1}
I20260619 20:34:15.802580 17681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:15.790310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:45943 (local address 127.0.0.1:40104)
0619 20:34:15.791316 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:15.791324 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:15.791352 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:15.792898 (+ 1546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:15.792898 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:15.792901 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:34:15.792979 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:15.792981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:15.797992 (+ 5011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:15.797993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:15.799171 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:15.799175 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:15.802343 (+ 3168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:15.802344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:15.802416 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:15.802423 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 20:34:15.802452 (+ 29us) client_negotiation.cc:240] Negotiation successful
0619 20:34:15.802498 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":995,"thread_start_us":74,"threads_started":1}
I20260619 20:34:15.802716 17678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:15.790074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:44779 (local address 127.0.0.1:51614)
0619 20:34:15.790982 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:15.791025 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:15.791083 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:15.799118 (+ 8035us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:15.799119 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:15.799122 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:34:15.799181 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:15.799183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:15.800501 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:15.800502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:15.801585 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:15.801588 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:15.802492 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:15.802493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:15.802574 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:15.802578 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:34:15.802605 (+ 27us) client_negotiation.cc:240] Negotiation successful
0619 20:34:15.802645 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":88,"threads_started":1}
I20260619 20:34:15.803175 17683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:15.790508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40104 (local address 127.4.83.65:45943)
0619 20:34:15.791635 (+ 1127us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:15.791639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:15.791647 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:34:15.791678 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:15.791680 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:15.791693 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:15.793056 (+ 1363us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:34:15.793173 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:15.795300 (+ 2127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:15.799279 (+ 3979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:15.800128 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:15.802917 (+ 2789us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:15.803005 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:15.803031 (+ 26us) server_negotiation.cc:299] Negotiation successful
0619 20:34:15.803083 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1096,"thread_start_us":51,"threads_started":1}
I20260619 20:34:15.803265 17680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:15.790189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51614 (local address 127.4.83.66:44779)
0619 20:34:15.798907 (+ 8718us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:15.798914 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:15.798930 (+ 16us) server_negotiation.cc:407] Connection header received
0619 20:34:15.798969 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:15.798971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:15.798988 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:15.799252 (+ 264us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:34:15.799338 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:15.800399 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:15.801689 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:15.802398 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:15.802783 (+ 385us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:15.803123 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:15.803149 (+ 26us) server_negotiation.cc:299] Negotiation successful
0619 20:34:15.803192 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8671,"thread_start_us":58,"threads_started":1}
I20260619 20:34:15.804013 17449 tablet_service.cc:1467] Processing CreateTablet for tablet 65afd44d9f1c4ce2a1a23a3e1c70b21c (DEFAULT_TABLE table=test-table [id=38dc7e1b841b45838fd55a0ced04042c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:34:15.804387 17449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65afd44d9f1c4ce2a1a23a3e1c70b21c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:34:15.804337 17314 tablet_service.cc:1467] Processing CreateTablet for tablet 65afd44d9f1c4ce2a1a23a3e1c70b21c (DEFAULT_TABLE table=test-table [id=38dc7e1b841b45838fd55a0ced04042c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:34:15.804651 17314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65afd44d9f1c4ce2a1a23a3e1c70b21c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:34:15.808581 17584 tablet_service.cc:1467] Processing CreateTablet for tablet 65afd44d9f1c4ce2a1a23a3e1c70b21c (DEFAULT_TABLE table=test-table [id=38dc7e1b841b45838fd55a0ced04042c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:34:15.808954 17584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65afd44d9f1c4ce2a1a23a3e1c70b21c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:34:15.812016 17684 tablet_bootstrap.cc:492] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219: Bootstrap starting.
I20260619 20:34:15.812716 17684 tablet_bootstrap.cc:654] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219: Neither blocks nor log segments found. Creating new log.
I20260619 20:34:15.813033 17684 log.cc:824] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219: Log is configured to *not* fsync() on all Append() calls
I20260619 20:34:15.814093 17686 tablet_bootstrap.cc:492] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 770401e91720425c8821923b5db0f43e: Bootstrap starting.
I20260619 20:34:15.814896 17686 tablet_bootstrap.cc:654] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 770401e91720425c8821923b5db0f43e: Neither blocks nor log segments found. Creating new log.
I20260619 20:34:15.815302 17686 log.cc:824] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 770401e91720425c8821923b5db0f43e: Log is configured to *not* fsync() on all Append() calls
I20260619 20:34:15.816437 17686 tablet_bootstrap.cc:492] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 770401e91720425c8821923b5db0f43e: No bootstrap required, opened a new log
I20260619 20:34:15.816512 17686 ts_tablet_manager.cc:1397] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 770401e91720425c8821923b5db0f43e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:34:15.818805 17686 raft_consensus.cc:348] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 770401e91720425c8821923b5db0f43e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f73e27c48c475fa6240020b1174219" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 44779 } } peers { permanent_uuid: "770401e91720425c8821923b5db0f43e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 45943 } } peers { permanent_uuid: "10829b22afa1485988e14740ab83245f" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45943 } }
I20260619 20:34:15.818974 17686 raft_consensus.cc:374] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 770401e91720425c8821923b5db0f43e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:34:15.819003 17686 raft_consensus.cc:729] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 770401e91720425c8821923b5db0f43e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 770401e91720425c8821923b5db0f43e, State: Initialized, Role: FOLLOWER
I20260619 20:34:15.819120 17686 consensus_queue.cc:260] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 770401e91720425c8821923b5db0f43e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f73e27c48c475fa6240020b1174219" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 44779 } } peers { permanent_uuid: "770401e91720425c8821923b5db0f43e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 45943 } } peers { permanent_uuid: "10829b22afa1485988e14740ab83245f" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45943 } }
I20260619 20:34:15.819387 17686 ts_tablet_manager.cc:1428] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 770401e91720425c8821923b5db0f43e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:34:15.820227 17651 heartbeater.cc:503] Master 127.4.83.126:39605 was elected leader, sending a full tablet report...
I20260619 20:34:15.820269 17684 tablet_bootstrap.cc:492] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219: No bootstrap required, opened a new log
I20260619 20:34:15.820333 17684 ts_tablet_manager.cc:1397] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260619 20:34:15.820782 17685 tablet_bootstrap.cc:492] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 10829b22afa1485988e14740ab83245f: Bootstrap starting.
I20260619 20:34:15.821477 17685 tablet_bootstrap.cc:654] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 10829b22afa1485988e14740ab83245f: Neither blocks nor log segments found. Creating new log.
I20260619 20:34:15.823346 17684 raft_consensus.cc:348] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f73e27c48c475fa6240020b1174219" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 44779 } } peers { permanent_uuid: "770401e91720425c8821923b5db0f43e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 45943 } } peers { permanent_uuid: "10829b22afa1485988e14740ab83245f" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45943 } }
I20260619 20:34:15.823482 17684 raft_consensus.cc:374] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:34:15.823513 17684 raft_consensus.cc:729] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f73e27c48c475fa6240020b1174219, State: Initialized, Role: FOLLOWER
I20260619 20:34:15.823849 17685 log.cc:824] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 10829b22afa1485988e14740ab83245f: Log is configured to *not* fsync() on all Append() calls
I20260619 20:34:15.824250 17684 consensus_queue.cc:260] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f73e27c48c475fa6240020b1174219" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 44779 } } peers { permanent_uuid: "770401e91720425c8821923b5db0f43e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 45943 } } peers { permanent_uuid: "10829b22afa1485988e14740ab83245f" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45943 } }
I20260619 20:34:15.824581 17684 ts_tablet_manager.cc:1428] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260619 20:34:15.824728 17516 heartbeater.cc:503] Master 127.4.83.126:39605 was elected leader, sending a full tablet report...
I20260619 20:34:15.825454 17685 tablet_bootstrap.cc:492] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 10829b22afa1485988e14740ab83245f: No bootstrap required, opened a new log
I20260619 20:34:15.825569 17685 ts_tablet_manager.cc:1397] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 10829b22afa1485988e14740ab83245f: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260619 20:34:15.827669 17685 raft_consensus.cc:348] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 10829b22afa1485988e14740ab83245f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f73e27c48c475fa6240020b1174219" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 44779 } } peers { permanent_uuid: "770401e91720425c8821923b5db0f43e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 45943 } } peers { permanent_uuid: "10829b22afa1485988e14740ab83245f" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45943 } }
I20260619 20:34:15.827863 17685 raft_consensus.cc:374] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 10829b22afa1485988e14740ab83245f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:34:15.827927 17685 raft_consensus.cc:729] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 10829b22afa1485988e14740ab83245f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10829b22afa1485988e14740ab83245f, State: Initialized, Role: FOLLOWER
I20260619 20:34:15.828053 17685 consensus_queue.cc:260] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 10829b22afa1485988e14740ab83245f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f73e27c48c475fa6240020b1174219" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 44779 } } peers { permanent_uuid: "770401e91720425c8821923b5db0f43e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 45943 } } peers { permanent_uuid: "10829b22afa1485988e14740ab83245f" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45943 } }
I20260619 20:34:15.828336 17685 ts_tablet_manager.cc:1428] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 10829b22afa1485988e14740ab83245f: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260619 20:34:15.828365 17381 heartbeater.cc:503] Master 127.4.83.126:39605 was elected leader, sending a full tablet report...
W20260619 20:34:15.869814 17382 tablet.cc:2369] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 10829b22afa1485988e14740ab83245f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 19 20:34:15 dist-test-slave-4m58 krb5kdc[16176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901255, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for HTTP/127.4.83.84@KRBTEST.COM
W20260619 20:34:15.998759 17652 tablet.cc:2369] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 770401e91720425c8821923b5db0f43e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:34:16.007928 17691 raft_consensus.cc:482] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:34:16.008088 17691 raft_consensus.cc:504] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f73e27c48c475fa6240020b1174219" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 44779 } } peers { permanent_uuid: "770401e91720425c8821923b5db0f43e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 45943 } } peers { permanent_uuid: "10829b22afa1485988e14740ab83245f" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45943 } }
I20260619 20:34:16.008491 17691 leader_election.cc:288] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 770401e91720425c8821923b5db0f43e (127.4.83.67:45943), 10829b22afa1485988e14740ab83245f (127.4.83.65:45943)
I20260619 20:34:16.014384 17683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:16.009084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:58429 (local address 127.4.83.65:45943)
0619 20:34:16.009153 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:16.009155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:16.009546 (+ 391us) server_negotiation.cc:407] Connection header received
0619 20:34:16.009633 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:16.009634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:16.009644 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:16.009669 (+ 25us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:34:16.009862 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:16.010597 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:16.013327 (+ 2730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:16.013846 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:16.013890 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:16.013901 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:16.014254 (+ 353us) server_negotiation.cc:299] Negotiation successful
0619 20:34:16.014305 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 20:34:16.014858 17695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:16.008808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:45943 (local address 127.4.83.66:58429)
0619 20:34:16.009494 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:16.009503 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:16.009571 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:16.009728 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:16.009729 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:16.009730 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:34:16.009794 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:16.009796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:16.011070 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:16.011071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:16.013238 (+ 2167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:16.013242 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:16.014105 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:16.014106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:16.014181 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:16.014185 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:34:16.014713 (+ 528us) client_negotiation.cc:240] Negotiation successful
0619 20:34:16.014758 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":89,"threads_started":1}
I20260619 20:34:16.015280 17682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:16.009002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:43625 (local address 127.4.83.67:45943)
0619 20:34:16.009137 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:16.009139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:16.009147 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:34:16.009176 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:16.009176 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:16.009188 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:16.009237 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:34:16.009885 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:16.010611 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:16.014062 (+ 3451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:16.014661 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:16.014709 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:16.014718 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:16.015147 (+ 429us) server_negotiation.cc:299] Negotiation successful
0619 20:34:16.015209 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260619 20:34:16.015506 17334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65afd44d9f1c4ce2a1a23a3e1c70b21c" candidate_uuid: "b1f73e27c48c475fa6240020b1174219" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10829b22afa1485988e14740ab83245f" is_pre_election: true
I20260619 20:34:16.015689 17334 raft_consensus.cc:2454] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 10829b22afa1485988e14740ab83245f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1f73e27c48c475fa6240020b1174219 in term 0.
I20260619 20:34:16.015945 17405 leader_election.cc:302] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10829b22afa1485988e14740ab83245f, b1f73e27c48c475fa6240020b1174219; no voters:
I20260619 20:34:16.016089 17691 raft_consensus.cc:2781] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:34:16.016127 17691 raft_consensus.cc:482] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:34:16.016146 17691 raft_consensus.cc:3037] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:34:16.016252 17694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:16.008666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:45943 (local address 127.4.83.66:43625)
0619 20:34:16.008919 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:16.008930 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:16.008985 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:16.009260 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:16.009261 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:16.009264 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:34:16.009353 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:16.009355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:16.010732 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:16.010733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:16.013950 (+ 3217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:16.013953 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:16.014986 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:16.014987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:16.015064 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:16.015068 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:34:16.016086 (+ 1018us) client_negotiation.cc:240] Negotiation successful
0619 20:34:16.016131 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":110,"threads_started":1}
I20260619 20:34:16.017015 17691 raft_consensus.cc:504] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f73e27c48c475fa6240020b1174219" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 44779 } } peers { permanent_uuid: "770401e91720425c8821923b5db0f43e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 45943 } } peers { permanent_uuid: "10829b22afa1485988e14740ab83245f" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45943 } }
I20260619 20:34:16.017251 17691 leader_election.cc:288] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219 [CANDIDATE]: Term 1 election: Requested vote from peers 770401e91720425c8821923b5db0f43e (127.4.83.67:45943), 10829b22afa1485988e14740ab83245f (127.4.83.65:45943)
I20260619 20:34:16.017527 17592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65afd44d9f1c4ce2a1a23a3e1c70b21c" candidate_uuid: "b1f73e27c48c475fa6240020b1174219" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770401e91720425c8821923b5db0f43e"
I20260619 20:34:16.017535 17591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65afd44d9f1c4ce2a1a23a3e1c70b21c" candidate_uuid: "b1f73e27c48c475fa6240020b1174219" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770401e91720425c8821923b5db0f43e" is_pre_election: true
I20260619 20:34:16.017652 17592 raft_consensus.cc:3037] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 770401e91720425c8821923b5db0f43e [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:34:16.018508 17592 raft_consensus.cc:2454] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 770401e91720425c8821923b5db0f43e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f73e27c48c475fa6240020b1174219 in term 1.
I20260619 20:34:16.019346 17403 leader_election.cc:302] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 770401e91720425c8821923b5db0f43e, b1f73e27c48c475fa6240020b1174219; no voters:
I20260619 20:34:16.019610 17334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65afd44d9f1c4ce2a1a23a3e1c70b21c" candidate_uuid: "b1f73e27c48c475fa6240020b1174219" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10829b22afa1485988e14740ab83245f"
I20260619 20:34:16.019703 17334 raft_consensus.cc:3037] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 10829b22afa1485988e14740ab83245f [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:34:16.020205 17691 raft_consensus.cc:2781] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:34:16.020377 17691 raft_consensus.cc:686] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219 [term 1 LEADER]: Becoming Leader. State: Replica: b1f73e27c48c475fa6240020b1174219, State: Running, Role: LEADER
I20260619 20:34:16.020462 17691 consensus_queue.cc:237] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f73e27c48c475fa6240020b1174219" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 44779 } } peers { permanent_uuid: "770401e91720425c8821923b5db0f43e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 45943 } } peers { permanent_uuid: "10829b22afa1485988e14740ab83245f" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45943 } }
I20260619 20:34:16.020607 17334 raft_consensus.cc:2454] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 10829b22afa1485988e14740ab83245f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1f73e27c48c475fa6240020b1174219 in term 1.
I20260619 20:34:16.021437 17180 catalog_manager.cc:5495] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1f73e27c48c475fa6240020b1174219 (127.4.83.66). New cstate: current_term: 1 leader_uuid: "b1f73e27c48c475fa6240020b1174219" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1f73e27c48c475fa6240020b1174219" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 44779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "770401e91720425c8821923b5db0f43e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 45943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10829b22afa1485988e14740ab83245f" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45943 } health_report { overall_health: UNKNOWN } } }
I20260619 20:34:16.030347 17680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:16.027008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51628 (local address 127.4.83.66:44779)
0619 20:34:16.027082 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:16.027084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:16.027093 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:34:16.027127 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:16.027128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:16.027165 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:16.027233 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:34:16.027394 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:16.028176 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:16.029278 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:16.029551 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:16.029612 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:16.029798 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:34:16.029928 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:34:16.030001 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:16.030239 (+ 238us) server_negotiation.cc:299] Negotiation successful
0619 20:34:16.030262 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 20:34:16.033408 17334 raft_consensus.cc:1261] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 10829b22afa1485988e14740ab83245f [term 1 FOLLOWER]: Refusing update from remote peer b1f73e27c48c475fa6240020b1174219: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:34:16.033859 17592 raft_consensus.cc:1261] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 770401e91720425c8821923b5db0f43e [term 1 FOLLOWER]: Refusing update from remote peer b1f73e27c48c475fa6240020b1174219: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:34:16.034088 17696 consensus_queue.cc:1035] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10829b22afa1485988e14740ab83245f" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 45943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:34:16.034433 17696 consensus_queue.cc:1035] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "770401e91720425c8821923b5db0f43e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 45943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:34:16.040292 17699 mvcc.cc:205] Tried to move back new op lower bound from 7298667544708636672 to 7298667544660209664. Current Snapshot: MvccSnapshot[applied={T|T < 7298667544708636672}]
I20260619 20:34:16.040452 17701 mvcc.cc:205] Tried to move back new op lower bound from 7298667544708636672 to 7298667544660209664. Current Snapshot: MvccSnapshot[applied={T|T < 7298667544708636672}]
I20260619 20:34:16.041839 17700 mvcc.cc:205] Tried to move back new op lower bound from 7298667544708636672 to 7298667544660209664. Current Snapshot: MvccSnapshot[applied={T|T < 7298667544708636672}]
I20260619 20:34:16.043999 17180 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:34:16.044114 17180 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:34:16.044824 17180 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb4877a8cc1a493cb445de67acc67e42: Sending DeleteTablet for 3 replicas of tablet 65afd44d9f1c4ce2a1a23a3e1c70b21c
I20260619 20:34:16.045174 17449 tablet_service.cc:1514] Processing DeleteTablet for tablet 65afd44d9f1c4ce2a1a23a3e1c70b21c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:34:16 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:51614
I20260619 20:34:16.045166 17584 tablet_service.cc:1514] Processing DeleteTablet for tablet 65afd44d9f1c4ce2a1a23a3e1c70b21c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:34:16 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:53080
I20260619 20:34:16.045171 17314 tablet_service.cc:1514] Processing DeleteTablet for tablet 65afd44d9f1c4ce2a1a23a3e1c70b21c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:34:16 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:40104
I20260619 20:34:16.045672 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 17219
I20260619 20:34:16.052983 17711 tablet_replica.cc:330] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219: stopping tablet replica
I20260619 20:34:16.053258 17711 raft_consensus.cc:2229] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:34:16.053545 17711 raft_consensus.cc:2258] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:34:16.054287 17711 ts_tablet_manager.cc:1905] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:34:16.057261 17710 tablet_replica.cc:330] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 770401e91720425c8821923b5db0f43e: stopping tablet replica
I20260619 20:34:16.057411 17710 raft_consensus.cc:2229] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 770401e91720425c8821923b5db0f43e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:34:16.057529 17710 raft_consensus.cc:2258] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 770401e91720425c8821923b5db0f43e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:34:16.060232 17710 ts_tablet_manager.cc:1905] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 770401e91720425c8821923b5db0f43e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:34:16.063975 17711 ts_tablet_manager.cc:1918] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:34:16.064095 17711 log.cc:1196] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-1/wal/wals/65afd44d9f1c4ce2a1a23a3e1c70b21c
I20260619 20:34:16.064378 17711 ts_tablet_manager.cc:1939] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P b1f73e27c48c475fa6240020b1174219: Deleting consensus metadata
I20260619 20:34:16.064879 17163 catalog_manager.cc:4841] TS b1f73e27c48c475fa6240020b1174219 (127.4.83.66:44779): tablet 65afd44d9f1c4ce2a1a23a3e1c70b21c (table test-table [id=38dc7e1b841b45838fd55a0ced04042c]) successfully deleted
I20260619 20:34:16.067155 17710 ts_tablet_manager.cc:1918] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 770401e91720425c8821923b5db0f43e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:34:16.067250 17710 log.cc:1196] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 770401e91720425c8821923b5db0f43e: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901102012824-4429-0/minicluster-data/ts-2/wal/wals/65afd44d9f1c4ce2a1a23a3e1c70b21c
I20260619 20:34:16.067485 17710 ts_tablet_manager.cc:1939] T 65afd44d9f1c4ce2a1a23a3e1c70b21c P 770401e91720425c8821923b5db0f43e: Deleting consensus metadata
W20260619 20:34:16.068145 17166 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.83.65:45943 (error 108)
I20260619 20:34:16.070875 17164 catalog_manager.cc:4841] TS 770401e91720425c8821923b5db0f43e (127.4.83.67:45943): tablet 65afd44d9f1c4ce2a1a23a3e1c70b21c (table test-table [id=38dc7e1b841b45838fd55a0ced04042c]) successfully deleted
I20260619 20:34:16.071224 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 17385
I20260619 20:34:16.073016 17681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:16.072843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:45943 (local address 127.0.0.1:40116)
0619 20:34:16.072914 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:16.072968 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.65:45943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 20:34:16.073263 17166 catalog_manager.cc:4584] TS 10829b22afa1485988e14740ab83245f (127.4.83.65:45943): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 65afd44d9f1c4ce2a1a23a3e1c70b21c: Network error: Client connection negotiation failed: client connection to 127.4.83.65:45943: connect: Connection refused (error 111)
I20260619 20:34:16.076812 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 17519
I20260619 20:34:16.087538 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 17146
I20260619 20:34:16.095319 4429 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 20:34:16.440202 4429 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 20:34:16.440310 4429 mini_ranger.cc:66] Stopping Ranger...
I20260619 20:34:16.786455 4429 mini_ranger.cc:68] Stopped Ranger
2026-06-19 20:34:16.786 UTC [16206] LOG: received smart shutdown request
2026-06-19 20:34:16.791 UTC [16206] LOG: background worker "logical replication launcher" (PID 16215) exited with exit code 1
2026-06-19 20:34:16.792 UTC [16210] LOG: shutting down
2026-06-19 20:34:16.983 UTC [16206] LOG: database system is shut down
2026-06-19T20:34:16Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50075 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:34:17 dist-test-slave-4m58 krb5kdc[17722](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:34:17 dist-test-slave-4m58 krb5kdc[17722](info): set up 2 sockets
Jun 19 20:34:17 dist-test-slave-4m58 krb5kdc[17722](info): commencing operation
krb5kdc: starting...
W20260619 20:34:18.076663 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.974s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:34:18 dist-test-slave-4m58 krb5kdc[17722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:34:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:34:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
I20260619 20:34:18.191563 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:32917
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:39217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:32917,127.4.83.125:43911,127.4.83.124:37241
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:34:18.270313 17735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:34:18.270534 17735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:34:18.270555 17735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:34:18.271689 17735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:34:18.271741 17735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:34:18.271754 17735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:34:18.271765 17735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:34:18.271775 17735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:34:18.273752 17735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:39217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:32917,127.4.83.125:43911,127.4.83.124:37241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:32917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:34:18.274025 17735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:34:18.274267 17735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:34:18.278949 17740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:18.278977 17741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:18.279009 17743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:18.279134 17735 server_base.cc:1034] running on GCE node
I20260619 20:34:18.279371 17735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:34:18.279680 17735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:34:18.280848 17735 hybrid_clock.cc:648] HybridClock initialized: now 1781901258280830 us; error 32 us; skew 500 ppm
Jun 19 20:34:18 dist-test-slave-4m58 krb5kdc[17722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901258, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:34:18.286005 17735 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:34:18.286435 17735 webserver.cc:460] Webserver started at http://127.4.83.126:46069/ using document root <none> and password file <none>
I20260619 20:34:18.286670 17735 fs_manager.cc:359] Metadata directory not provided
I20260619 20:34:18.286718 17735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:34:18.286844 17735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:34:18.287909 17735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b"
format_stamp: "Formatted at 2026-06-19 20:34:18 on dist-test-slave-4m58"
server_key: "7f7ea3960b56dfdef85ba79c4a5c756a"
server_key_iv: "cb4960dac850e0488cae2d05b92a3e94"
server_key_version: "encryptionkey@0"
I20260619 20:34:18.288242 17735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b"
format_stamp: "Formatted at 2026-06-19 20:34:18 on dist-test-slave-4m58"
server_key: "7f7ea3960b56dfdef85ba79c4a5c756a"
server_key_iv: "cb4960dac850e0488cae2d05b92a3e94"
server_key_version: "encryptionkey@0"
I20260619 20:34:18.289512 17735 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:34:18.290186 17750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:18.290360 17735 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:34:18.290444 17735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b"
format_stamp: "Formatted at 2026-06-19 20:34:18 on dist-test-slave-4m58"
server_key: "7f7ea3960b56dfdef85ba79c4a5c756a"
server_key_iv: "cb4960dac850e0488cae2d05b92a3e94"
server_key_version: "encryptionkey@0"
I20260619 20:34:18.290518 17735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:34:18.301211 17735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:34:18.303313 17735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:34:18.303452 17735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:34:18.307220 17735 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:32917
I20260619 20:34:18.307269 17803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:32917 every 8 connection(s)
I20260619 20:34:18.307572 17735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:34:18.308421 17804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:34:18.309968 17804 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.126" port: 32917 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 20:34:18 dist-test-slave-4m58 krb5kdc[17722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901258, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:34:18.316459 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 17735
I20260619 20:34:18.316591 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:34:18.316802 4429 external_mini_cluster.cc:1428] Setting key 555489bc217cf5f4d2718db660765f40
I20260619 20:34:18.321323 17807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.310696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47762 (local address 127.4.83.126:32917)
0619 20:34:18.310874 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:18.310883 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:18.311033 (+ 150us) server_negotiation.cc:407] Connection header received
0619 20:34:18.311588 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:18.311590 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:18.311607 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:18.311680 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:18.312311 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.313119 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.313934 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.314231 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.314323 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:18.316393 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:18.316400 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:18.316400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:18.316407 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:18.319258 (+ 2851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:18.319788 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:18.319791 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:18.319791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:18.319955 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:18.320308 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:18.320309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:18.320309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:18.320592 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:18.320667 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:18.320990 (+ 323us) server_negotiation.cc:299] Negotiation successful
0619 20:34:18.321148 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":128,"thread_start_us":64,"threads_started":1}
I20260619 20:34:18.321332 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.310624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:32917 (local address 127.0.0.1:47762)
0619 20:34:18.310863 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:18.310906 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:18.311441 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:18.311696 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:18.311698 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:18.312061 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:34:18.312222 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:18.312227 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.313201 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.313202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:18.313863 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:18.313865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.314298 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.314299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:18.314362 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:18.314865 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:34:18.314874 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:34:18.316309 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:34:18.319380 (+ 3071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:18.319390 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:18.319393 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:18.319682 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:18.320071 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:18.320072 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:18.320073 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:18.320194 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:18.320669 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:34:18.320676 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:34:18.320757 (+ 81us) client_negotiation.cc:769] Sending connection context
0619 20:34:18.320865 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 20:34:18.321072 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":96,"threads_started":1}
I20260619 20:34:18.322139 17804 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.125" port: 43911 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:34:18.322522 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.322384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:43911 (local address 127.0.0.1:36612)
0619 20:34:18.322431 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:18.322494 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.125:43911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 20:34:18.322666 17754 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.83.125:43911: connect: Connection refused (error 111)
I20260619 20:34:18.323078 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.322983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:43911 (local address 127.0.0.1:36622)
0619 20:34:18.323026 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:18.323056 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.125:43911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260619 20:34:18.323184 17804 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.83.125:43911: Network error: Client connection negotiation failed: client connection to 127.4.83.125:43911: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.4.83.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.125.keytab.
Entry for principal kudu/127.4.83.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.125.keytab.
I20260619 20:34:18.340804 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.125:43911
--webserver_interface=127.4.83.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.125.keytab
--principal=kudu/127.4.83.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:39217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:32917,127.4.83.125:43911,127.4.83.124:37241
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:34:18.372408 17804 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.83.125" port: 43911 } attempt: 1
I20260619 20:34:18.372958 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.372790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:43911 (local address 127.0.0.1:36630)
0619 20:34:18.372861 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:18.372909 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.125:43911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260619 20:34:18.373274 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.373188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:43911 (local address 127.0.0.1:36644)
0619 20:34:18.373228 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:18.373255 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.125:43911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 20:34:18.373379 17804 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.83.125:43911: Network error: Client connection negotiation failed: client connection to 127.4.83.125:43911: connect: Connection refused (error 111)
W20260619 20:34:18.421029 17811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:34:18.421245 17811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:34:18.421267 17811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:34:18.422384 17811 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:34:18.422427 17811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:34:18.422439 17811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:34:18.422451 17811 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:34:18.422461 17811 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:34:18.423972 17811 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:39217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:32917,127.4.83.125:43911,127.4.83.124:37241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.125.keytab
--principal=kudu/127.4.83.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.125:43911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.83.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:34:18.424211 17811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:34:18.424414 17811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:34:18.429246 17811 server_base.cc:1034] running on GCE node
W20260619 20:34:18.429216 17817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:18.429276 17816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:18.429216 17819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:18.429706 17811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:34:18.429987 17811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:34:18.431135 17811 hybrid_clock.cc:648] HybridClock initialized: now 1781901258431123 us; error 28 us; skew 500 ppm
Jun 19 20:34:18 dist-test-slave-4m58 krb5kdc[17722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901258, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:34:18.433796 17811 init.cc:377] Logged in from keytab as kudu/127.4.83.125@KRBTEST.COM (short username kudu)
I20260619 20:34:18.434253 17811 webserver.cc:460] Webserver started at http://127.4.83.125:38809/ using document root <none> and password file <none>
I20260619 20:34:18.434453 17811 fs_manager.cc:359] Metadata directory not provided
I20260619 20:34:18.434495 17811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:34:18.434603 17811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:34:18.435670 17811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-1/data/instance:
uuid: "0129451a23974a11832e01fbaed0a60f"
format_stamp: "Formatted at 2026-06-19 20:34:18 on dist-test-slave-4m58"
server_key: "9a74938590a654ba59d35d147e7bdaa0"
server_key_iv: "e003167b2a6f6996db4e5f13fa950afd"
server_key_version: "encryptionkey@0"
I20260619 20:34:18.436026 17811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-1/wal/instance:
uuid: "0129451a23974a11832e01fbaed0a60f"
format_stamp: "Formatted at 2026-06-19 20:34:18 on dist-test-slave-4m58"
server_key: "9a74938590a654ba59d35d147e7bdaa0"
server_key_iv: "e003167b2a6f6996db4e5f13fa950afd"
server_key_version: "encryptionkey@0"
I20260619 20:34:18.437302 17811 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:34:18.438045 17826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:18.438252 17811 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:34:18.438344 17811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-1/wal
uuid: "0129451a23974a11832e01fbaed0a60f"
format_stamp: "Formatted at 2026-06-19 20:34:18 on dist-test-slave-4m58"
server_key: "9a74938590a654ba59d35d147e7bdaa0"
server_key_iv: "e003167b2a6f6996db4e5f13fa950afd"
server_key_version: "encryptionkey@0"
I20260619 20:34:18.438400 17811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:34:18.441552 17804 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.83.125" port: 43911 } attempt: 2
I20260619 20:34:18.441985 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.441849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:43911 (local address 127.0.0.1:36654)
0619 20:34:18.441904 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:18.441948 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.125:43911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 20:34:18.442353 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.442260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:43911 (local address 127.0.0.1:36660)
0619 20:34:18.442300 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:18.442328 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.125:43911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260619 20:34:18.442435 17804 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.83.125:43911: Network error: Client connection negotiation failed: client connection to 127.4.83.125:43911: connect: Connection refused (error 111)
I20260619 20:34:18.458645 17811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:34:18.460750 17811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:34:18.460891 17811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:34:18.464905 17811 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.125:43911
I20260619 20:34:18.465055 17878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.125:43911 every 8 connection(s)
I20260619 20:34:18.465323 17811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-1/data/info.pb
I20260619 20:34:18.466231 17879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:34:18.467728 17879 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.126" port: 32917 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 20:34:18 dist-test-slave-4m58 krb5kdc[17722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901258, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.125@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:34:18.475595 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 17811
I20260619 20:34:18.475716 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-1/wal/instance
I20260619 20:34:18.475948 4429 external_mini_cluster.cc:1428] Setting key b05eb9afba8c7e9073f9773e5451f08a
I20260619 20:34:18.476646 17807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.468491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47766 (local address 127.4.83.126:32917)
0619 20:34:18.468542 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:18.468544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:18.468840 (+ 296us) server_negotiation.cc:407] Connection header received
0619 20:34:18.469212 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:18.469213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:18.469225 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:18.469288 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:18.469894 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.470422 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.471392 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.471603 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.471657 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:18.473740 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:18.473742 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:18.473742 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:18.473751 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:18.475102 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:18.475617 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:18.475618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:18.475618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:18.475679 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:18.475993 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:18.475995 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:18.475995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:18.476158 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:18.476229 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:18.476471 (+ 242us) server_negotiation.cc:299] Negotiation successful
0619 20:34:18.476560 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 20:34:18.476845 17881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.468439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:32917 (local address 127.0.0.1:47766)
0619 20:34:18.468599 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:18.468642 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:18.469119 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:18.469341 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:18.469344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:18.469597 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:34:18.469796 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:18.469800 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.470511 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.470512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:18.471313 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:18.471315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.471669 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.471670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:18.471769 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:18.472305 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:34:18.472313 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:34:18.473641 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:34:18.475188 (+ 1547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:18.475195 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:18.475197 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:18.475514 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:18.475767 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:18.475768 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:18.475769 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:18.475927 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:18.476222 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:34:18.476227 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:34:18.476355 (+ 128us) client_negotiation.cc:769] Sending connection context
0619 20:34:18.476442 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 20:34:18.476582 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":68,"threads_started":1}
I20260619 20:34:18.477429 17879 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.125" port: 43911 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 20:34:18 dist-test-slave-4m58 krb5kdc[17722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901258, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.125@KRBTEST.COM for kudu/127.4.83.125@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.83.124@KRBTEST.COM; defaulting to no policy
I20260619 20:34:18.484782 17883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.477769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36668 (local address 127.4.83.125:43911)
0619 20:34:18.477932 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:18.477936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:18.477952 (+ 16us) server_negotiation.cc:407] Connection header received
0619 20:34:18.477967 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:18.477969 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:18.477978 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:18.478037 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:18.478305 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.478984 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.479918 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.480228 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.480280 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:18.482192 (+ 1912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:18.482200 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:18.482200 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:18.482206 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:18.483670 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:18.484014 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:18.484016 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:18.484017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:18.484094 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:18.484290 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:18.484291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:18.484291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:18.484441 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:18.484504 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:18.484602 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 20:34:18.484695 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":140,"thread_start_us":102,"threads_started":1}
I20260619 20:34:18.484833 17881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.477673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:43911 (local address 127.0.0.1:36668)
0619 20:34:18.477712 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:18.477722 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:18.477753 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:18.478046 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:18.478047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:18.478181 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:34:18.478244 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:18.478246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.479057 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.479058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:18.479824 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:18.479826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.480294 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.480295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:18.480374 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:18.480745 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:34:18.480750 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:34:18.482109 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:34:18.483744 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:18.483745 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:18.483745 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:18.483954 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:18.484172 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:18.484173 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:18.484173 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:18.484234 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:18.484492 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:34:18.484493 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:34:18.484531 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 20:34:18.484584 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 20:34:18.484750 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":76}
I20260619 20:34:18.485436 17879 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.124" port: 37241 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:34:18.485764 17881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.485651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:37241 (local address 127.0.0.1:45510)
0619 20:34:18.485687 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:18.485740 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:37241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.124@KRBTEST.COM" created.
W20260619 20:34:18.485896 17828 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.83.124:37241: connect: Connection refused (error 111)
I20260619 20:34:18.486332 17881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.486254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:37241 (local address 127.0.0.1:45512)
0619 20:34:18.486293 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:18.486316 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:37241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 20:34:18.486429 17879 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.83.124:37241: Network error: Client connection negotiation failed: client connection to 127.4.83.124:37241: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.124.keytab.
Entry for principal kudu/127.4.83.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.124.keytab.
I20260619 20:34:18.500425 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.124:37241
--webserver_interface=127.4.83.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.124.keytab
--principal=kudu/127.4.83.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:39217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:32917,127.4.83.125:43911,127.4.83.124:37241
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:34:18.533597 17804 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.83.125" port: 43911 } attempt: 3
I20260619 20:34:18.535643 17879 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.83.124" port: 37241 } attempt: 1
I20260619 20:34:18.536100 17881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.535934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:37241 (local address 127.0.0.1:45514)
0619 20:34:18.536003 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:18.536056 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:37241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 20:34:18.536721 17881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.536553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:37241 (local address 127.0.0.1:45516)
0619 20:34:18.536647 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:18.536687 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:37241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260619 20:34:18.537001 17879 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.83.124:37241: Network error: Client connection negotiation failed: client connection to 127.4.83.124:37241: connect: Connection refused (error 111)
Jun 19 20:34:18 dist-test-slave-4m58 krb5kdc[17722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901258, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for kudu/127.4.83.125@KRBTEST.COM
I20260619 20:34:18.540956 17883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.534086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36678 (local address 127.4.83.125:43911)
0619 20:34:18.534145 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:18.534147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:18.534156 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:34:18.534183 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:18.534184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:18.534194 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:18.534242 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:18.534560 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.535078 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.535870 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.536126 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.536606 (+ 480us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:18.538490 (+ 1884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:18.538492 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:18.538492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:18.538500 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:18.539861 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:18.540203 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:18.540204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:18.540205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:18.540264 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:18.540428 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:18.540429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:18.540429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:18.540574 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:18.540634 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:18.540771 (+ 137us) server_negotiation.cc:299] Negotiation successful
0619 20:34:18.540856 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 20:34:18.540956 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.534003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:43911 (local address 127.0.0.1:36678)
0619 20:34:18.534072 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:18.534084 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:18.534131 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:18.534253 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:18.534254 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:18.534415 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:34:18.534482 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:18.534484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.535154 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.535155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:18.535742 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:18.535745 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.536231 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.536233 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:18.536402 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:18.536877 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:34:18.536883 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:34:18.538381 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:34:18.539958 (+ 1577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:18.539959 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:18.539959 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:18.540140 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:18.540319 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:18.540319 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:18.540320 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:18.540379 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:18.540631 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:34:18.540632 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:34:18.540684 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 20:34:18.540744 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 20:34:18.540855 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260619 20:34:18.541821 17804 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.124" port: 37241 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:34:18.542191 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.542083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:37241 (local address 127.0.0.1:45532)
0619 20:34:18.542123 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:18.542161 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:37241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260619 20:34:18.542477 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.542399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:37241 (local address 127.0.0.1:45546)
0619 20:34:18.542432 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:18.542455 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:37241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260619 20:34:18.542554 17804 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.83.124:37241: Network error: Client connection negotiation failed: client connection to 127.4.83.124:37241: connect: Connection refused (error 111)
I20260619 20:34:18.573704 17804 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.83.124" port: 37241 } attempt: 1
I20260619 20:34:18.574177 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.574051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:37241 (local address 127.0.0.1:45558)
0619 20:34:18.574110 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:18.574146 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:37241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 20:34:18.574620 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.574505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:37241 (local address 127.0.0.1:45564)
0619 20:34:18.574566 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:18.574593 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:37241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260619 20:34:18.574704 17804 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.83.124:37241: Network error: Client connection negotiation failed: client connection to 127.4.83.124:37241: connect: Connection refused (error 111)
W20260619 20:34:18.585482 17886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:34:18.585717 17886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:34:18.585749 17886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:34:18.586953 17886 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:34:18.587001 17886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:34:18.587020 17886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:34:18.587046 17886 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:34:18.587065 17886 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:34:18.588719 17886 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:39217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:32917,127.4.83.125:43911,127.4.83.124:37241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.124.keytab
--principal=kudu/127.4.83.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.124:37241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.83.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:34:18.588973 17886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:34:18.589200 17886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:34:18.594170 17892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:18.594321 17886 server_base.cc:1034] running on GCE node
W20260619 20:34:18.594158 17894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:18.594158 17891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:18.594810 17886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:34:18.595148 17886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:34:18.596328 17886 hybrid_clock.cc:648] HybridClock initialized: now 1781901258596311 us; error 28 us; skew 500 ppm
Jun 19 20:34:18 dist-test-slave-4m58 krb5kdc[17722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901258, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:34:18.599012 17886 init.cc:377] Logged in from keytab as kudu/127.4.83.124@KRBTEST.COM (short username kudu)
I20260619 20:34:18.599496 17886 webserver.cc:460] Webserver started at http://127.4.83.124:32863/ using document root <none> and password file <none>
I20260619 20:34:18.599717 17886 fs_manager.cc:359] Metadata directory not provided
I20260619 20:34:18.599766 17886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:34:18.599931 17886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:34:18.601009 17886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-2/data/instance:
uuid: "2e4d47fc9de94dabb94915678b001d54"
format_stamp: "Formatted at 2026-06-19 20:34:18 on dist-test-slave-4m58"
server_key: "a2231a6f72c1ea653142ffee324c63be"
server_key_iv: "0feec444d0f263ad7a15bf121614b0ae"
server_key_version: "encryptionkey@0"
I20260619 20:34:18.601361 17886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-2/wal/instance:
uuid: "2e4d47fc9de94dabb94915678b001d54"
format_stamp: "Formatted at 2026-06-19 20:34:18 on dist-test-slave-4m58"
server_key: "a2231a6f72c1ea653142ffee324c63be"
server_key_iv: "0feec444d0f263ad7a15bf121614b0ae"
server_key_version: "encryptionkey@0"
I20260619 20:34:18.602797 17886 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:34:18.603688 17901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:18.603890 17886 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:34:18.603993 17886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-2/wal
uuid: "2e4d47fc9de94dabb94915678b001d54"
format_stamp: "Formatted at 2026-06-19 20:34:18 on dist-test-slave-4m58"
server_key: "a2231a6f72c1ea653142ffee324c63be"
server_key_iv: "0feec444d0f263ad7a15bf121614b0ae"
server_key_version: "encryptionkey@0"
I20260619 20:34:18.604064 17886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:34:18.605182 17879 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.83.124" port: 37241 } attempt: 2
I20260619 20:34:18.605646 17881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.605497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:37241 (local address 127.0.0.1:45578)
0619 20:34:18.605553 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:18.605614 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:37241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 20:34:18.605973 17881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.605881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:37241 (local address 127.0.0.1:45582)
0619 20:34:18.605922 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:18.605948 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:37241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260619 20:34:18.606046 17879 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.83.124:37241: Network error: Client connection negotiation failed: client connection to 127.4.83.124:37241: connect: Connection refused (error 111)
I20260619 20:34:18.614375 17886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:34:18.616500 17886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:34:18.616633 17886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:34:18.620564 17886 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.124:37241
I20260619 20:34:18.620608 17954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.124:37241 every 8 connection(s)
I20260619 20:34:18.620947 17886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-2/data/info.pb
I20260619 20:34:18.621771 17955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:34:18.623344 17955 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.126" port: 32917 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:34:18.625296 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 17886
I20260619 20:34:18.625403 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901102012824-4429-0/minicluster-data/master-2/wal/instance
I20260619 20:34:18.625605 4429 external_mini_cluster.cc:1428] Setting key 8809304558ebc04f1b68d5c418664994
Jun 19 20:34:18 dist-test-slave-4m58 krb5kdc[17722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901258, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.124@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
Jun 19 20:34:18 dist-test-slave-4m58 krb5kdc[17722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.124@KRBTEST.COM
I20260619 20:34:18.632817 17807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.624087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47768 (local address 127.4.83.126:32917)
0619 20:34:18.624142 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:18.624144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:18.624400 (+ 256us) server_negotiation.cc:407] Connection header received
0619 20:34:18.624875 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:18.624876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:18.624897 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:18.624991 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:18.625768 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.626350 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.627312 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.627524 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.627581 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:18.629863 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:18.629865 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:18.629865 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:18.629872 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:18.631292 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:18.631714 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:18.631716 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:18.631716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:18.631779 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:18.632083 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:18.632084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:18.632084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:18.632229 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:18.632283 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:18.632662 (+ 379us) server_negotiation.cc:299] Negotiation successful
0619 20:34:18.632741 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 20:34:18.633023 17957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.624083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:32917 (local address 127.0.0.1:47768)
0619 20:34:18.624260 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:18.624296 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:18.624767 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:18.625129 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:18.625132 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:18.625410 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:34:18.625678 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:18.625684 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.626434 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.626435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:18.627227 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:18.627229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.627592 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.627593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:18.627727 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:18.628269 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:34:18.628278 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:34:18.629748 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:34:18.631382 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:18.631390 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:18.631392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:18.631627 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:18.631892 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:18.631893 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:18.631894 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:18.632018 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:18.632291 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:34:18.632297 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:34:18.632477 (+ 180us) client_negotiation.cc:769] Sending connection context
0619 20:34:18.632599 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 20:34:18.632764 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":73,"threads_started":1}
I20260619 20:34:18.633653 17955 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.125" port: 43911 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:34:18.634950 17959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.625963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45592 (local address 127.4.83.124:37241)
0619 20:34:18.626138 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:18.626142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:18.626165 (+ 23us) server_negotiation.cc:407] Connection header received
0619 20:34:18.626185 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:18.626187 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:18.626206 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:18.626269 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:18.626921 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.627636 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.628499 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.628804 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.628859 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:18.631707 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:18.631710 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:18.631711 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:18.631716 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:18.633540 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:18.634134 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:18.634137 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:18.634138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:18.634199 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:18.634426 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:18.634427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:18.634428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:18.634598 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:18.634724 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:18.634813 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 20:34:18.634876 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":111,"threads_started":1}
Jun 19 20:34:18 dist-test-slave-4m58 krb5kdc[17722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901258, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.124@KRBTEST.COM for kudu/127.4.83.125@KRBTEST.COM
I20260619 20:34:18.640691 17883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.634004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36688 (local address 127.4.83.125:43911)
0619 20:34:18.634051 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:18.634053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:18.634060 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:34:18.634083 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:18.634084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:18.634094 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:18.634145 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:18.634419 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.635092 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.636028 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.636321 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.636393 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:18.638111 (+ 1718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:18.638112 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:18.638113 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:18.638121 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:18.639659 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:18.640012 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:18.640013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:18.640013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:18.640067 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:18.640227 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:18.640228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:18.640228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:18.640355 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:18.640403 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:18.640561 (+ 158us) server_negotiation.cc:299] Negotiation successful
0619 20:34:18.640624 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:34:18.640734 17957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.633924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:43911 (local address 127.0.0.1:36688)
0619 20:34:18.633967 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:18.633977 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:18.634013 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:18.634148 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:18.634149 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:18.634287 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:34:18.634335 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:18.634336 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.635199 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.635200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:18.635954 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:18.635958 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.636404 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.636405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:18.636475 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:18.636839 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:34:18.636843 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:34:18.638032 (+ 1189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:34:18.639738 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:18.639738 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:18.639739 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:18.639951 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:18.640129 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:18.640129 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:18.640129 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:18.640179 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:18.640410 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:34:18.640411 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:34:18.640466 (+ 55us) client_negotiation.cc:769] Sending connection context
0619 20:34:18.640541 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 20:34:18.640634 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
Jun 19 20:34:18 dist-test-slave-4m58 krb5kdc[17722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:34:18.641124 17955 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.124" port: 37241 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:34:18.644168 17807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.635932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47770 (local address 127.4.83.126:32917)
0619 20:34:18.635978 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:18.635980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:18.635991 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:34:18.636016 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:18.636017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:18.636027 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:18.636072 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:18.636624 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.637295 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.638651 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.638846 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.638988 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:18.641442 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:18.641444 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:18.641444 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:18.641452 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:18.643093 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:18.643397 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:18.643398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:18.643398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:18.643460 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:18.643630 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:18.643630 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:18.643631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:18.643731 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:18.643946 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:18.643987 (+ 41us) server_negotiation.cc:299] Negotiation successful
0619 20:34:18.644096 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Jun 19 20:34:18 dist-test-slave-4m58 krb5kdc[17722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901258, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.124@KRBTEST.COM for kudu/127.4.83.124@KRBTEST.COM
I20260619 20:34:18.647869 17957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.641403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:37241 (local address 127.0.0.1:45596)
0619 20:34:18.641646 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:18.641654 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:18.641693 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:18.641797 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:18.641798 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:18.641911 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:34:18.641952 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:18.641953 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.642729 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.642730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:18.643381 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:18.643384 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.643793 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.643795 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:18.643866 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:18.644227 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:34:18.644232 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:34:18.645381 (+ 1149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:34:18.646785 (+ 1404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:18.646785 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:18.646785 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:18.646975 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:18.647161 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:18.647162 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:18.647162 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:18.647225 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:18.647484 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:34:18.647485 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:34:18.647529 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 20:34:18.647600 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 20:34:18.647691 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228}
I20260619 20:34:18.647879 17959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.641426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45596 (local address 127.4.83.124:37241)
0619 20:34:18.641466 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:18.641468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:18.641698 (+ 230us) server_negotiation.cc:407] Connection header received
0619 20:34:18.641723 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:18.641724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:18.641736 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:18.641790 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:18.642017 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.642636 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.643453 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.643727 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.643782 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:18.645447 (+ 1665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:18.645448 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:18.645448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:18.645455 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:18.646720 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:18.647038 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:18.647039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:18.647040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:18.647098 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:18.647295 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:18.647296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:18.647296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:18.647425 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:18.647486 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:18.647616 (+ 130us) server_negotiation.cc:299] Negotiation successful
0619 20:34:18.647734 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":21}
Jun 19 20:34:18 dist-test-slave-4m58 krb5kdc[17722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.125@KRBTEST.COM
I20260619 20:34:18.649557 17955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e4d47fc9de94dabb94915678b001d54: Bootstrap starting.
I20260619 20:34:18.649837 17804 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.83.124" port: 37241 } attempt: 2
I20260619 20:34:18.650542 17955 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e4d47fc9de94dabb94915678b001d54: Neither blocks nor log segments found. Creating new log.
I20260619 20:34:18.650843 17955 log.cc:824] T 00000000000000000000000000000000 P 2e4d47fc9de94dabb94915678b001d54: Log is configured to *not* fsync() on all Append() calls
I20260619 20:34:18.651808 17955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e4d47fc9de94dabb94915678b001d54: No bootstrap required, opened a new log
I20260619 20:34:18.652284 17883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.644779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36692 (local address 127.4.83.125:43911)
0619 20:34:18.644822 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:18.644824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:18.644830 (+ 6us) server_negotiation.cc:407] Connection header received
0619 20:34:18.644848 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:18.644849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:18.644857 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:18.644902 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:18.645356 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.646013 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.646742 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.647021 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.647087 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:18.649319 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:18.649320 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:18.649321 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:18.649327 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:18.650993 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:18.651350 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:18.651351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:18.651352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:18.651417 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:18.651623 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:18.651624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:18.651624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:18.651763 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:18.652063 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:18.652112 (+ 49us) server_negotiation.cc:299] Negotiation successful
0619 20:34:18.652198 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Jun 19 20:34:18 dist-test-slave-4m58 krb5kdc[17722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901258, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for kudu/127.4.83.124@KRBTEST.COM
I20260619 20:34:18.653942 4429 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:34:18.653832 17955 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e4d47fc9de94dabb94915678b001d54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 32917 } } peers { permanent_uuid: "0129451a23974a11832e01fbaed0a60f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 43911 } } peers { permanent_uuid: "2e4d47fc9de94dabb94915678b001d54" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 37241 } }
I20260619 20:34:18.653988 17955 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e4d47fc9de94dabb94915678b001d54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:34:18.654012 17955 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e4d47fc9de94dabb94915678b001d54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e4d47fc9de94dabb94915678b001d54, State: Initialized, Role: FOLLOWER
I20260619 20:34:18.654121 17955 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e4d47fc9de94dabb94915678b001d54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 32917 } } peers { permanent_uuid: "0129451a23974a11832e01fbaed0a60f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 43911 } } peers { permanent_uuid: "2e4d47fc9de94dabb94915678b001d54" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 37241 } }
I20260619 20:34:18.654362 17961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e4d47fc9de94dabb94915678b001d54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 32917 } } peers { permanent_uuid: "0129451a23974a11832e01fbaed0a60f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 43911 } } peers { permanent_uuid: "2e4d47fc9de94dabb94915678b001d54" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 37241 } } }
I20260619 20:34:18.654413 17955 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e4d47fc9de94dabb94915678b001d54 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:34:18.654430 17961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e4d47fc9de94dabb94915678b001d54 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 20:34:18.656032 17974 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2e4d47fc9de94dabb94915678b001d54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:34:18.656136 17974 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:34:18.656786 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.650107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:37241 (local address 127.0.0.1:45610)
0619 20:34:18.650162 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:18.650171 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:18.650208 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:18.650337 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:18.650337 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:18.650608 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:34:18.650667 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:18.650670 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.651417 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.651418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:18.652109 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:18.652113 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.652461 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.652462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:18.652514 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:18.652872 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:34:18.652877 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:34:18.654050 (+ 1173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:34:18.655656 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:18.655657 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:18.655658 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:18.655911 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:18.656125 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:18.656126 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:18.656126 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:18.656188 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:18.656506 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:34:18.656507 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:34:18.656555 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 20:34:18.656602 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 20:34:18.656694 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 20:34:18.657157 17959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.650157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45610 (local address 127.4.83.124:37241)
0619 20:34:18.650198 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:18.650200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:18.650209 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:34:18.650254 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:18.650254 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:18.650264 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:18.650319 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:18.650754 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.651341 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.652194 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.652395 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.652576 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:18.654130 (+ 1554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:18.654132 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:18.654132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:18.654139 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:18.655552 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:18.655997 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:18.655998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:18.655998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:18.656057 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:18.656256 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:18.656257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:18.656257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:18.656412 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:18.656963 (+ 551us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:18.657005 (+ 42us) server_negotiation.cc:299] Negotiation successful
0619 20:34:18.657078 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:34:18.658881 17804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b: Bootstrap starting.
I20260619 20:34:18.656574 17886 master_runner.cc:186] Error getting master registration for 127.4.83.126:32917: OK, instance_id { permanent_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" instance_seqno: 1781901258306124 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 20:34:18.659056 17886 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 20:34:18.659559 17804 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b: Neither blocks nor log segments found. Creating new log.
I20260619 20:34:18.659890 17804 log.cc:824] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b: Log is configured to *not* fsync() on all Append() calls
I20260619 20:34:18.660775 17804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b: No bootstrap required, opened a new log
I20260619 20:34:18.663851 17804 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 32917 } } peers { permanent_uuid: "0129451a23974a11832e01fbaed0a60f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 43911 } } peers { permanent_uuid: "2e4d47fc9de94dabb94915678b001d54" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 37241 } }
I20260619 20:34:18.664001 17804 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:34:18.664036 17804 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0cd06fe4bc94a5a9dd50132cbbcee0b, State: Initialized, Role: FOLLOWER
I20260619 20:34:18.664149 17804 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 32917 } } peers { permanent_uuid: "0129451a23974a11832e01fbaed0a60f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 43911 } } peers { permanent_uuid: "2e4d47fc9de94dabb94915678b001d54" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 37241 } }
I20260619 20:34:18.664507 17804 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:34:18.667716 17883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.657722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36702 (local address 127.4.83.125:43911)
0619 20:34:18.659976 (+ 2254us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:18.659977 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:18.659985 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:34:18.660004 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:18.660004 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:18.660013 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:18.660066 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:18.660777 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.661528 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.662537 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.662807 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.662846 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:18.664384 (+ 1538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:18.664385 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:18.664385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:18.664393 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:18.665912 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:18.666298 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:18.666299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:18.666300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:18.666357 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:18.666573 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:18.666573 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:18.666574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:18.666709 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:18.667315 (+ 606us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:18.667537 (+ 222us) server_negotiation.cc:299] Negotiation successful
0619 20:34:18.667622 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2178}
I20260619 20:34:18.667934 17807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.660124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47778 (local address 127.4.83.126:32917)
0619 20:34:18.660184 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:18.660185 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:18.660310 (+ 125us) server_negotiation.cc:407] Connection header received
0619 20:34:18.660398 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:18.660399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:18.660409 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:18.660436 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:18.661068 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.661709 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.662769 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.663413 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.663455 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:18.665294 (+ 1839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:18.665296 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:18.665296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:18.665304 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:18.666766 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:18.667141 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:18.667142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:18.667142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:18.667202 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:18.667393 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:18.667394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:18.667394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:18.667513 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:18.667573 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:18.667755 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 20:34:18.667825 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:34:18.667246 17735 master_runner.cc:186] Error getting master registration for 127.4.83.125:43911: OK, instance_id { permanent_uuid: "0129451a23974a11832e01fbaed0a60f" instance_seqno: 1781901258463720 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 20:34:18.670058 17735 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260619 20:34:18.671684 17995 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:34:18.671862 17995 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:34:18.672029 17984 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 32917 } } peers { permanent_uuid: "0129451a23974a11832e01fbaed0a60f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 43911 } } peers { permanent_uuid: "2e4d47fc9de94dabb94915678b001d54" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 37241 } } }
I20260619 20:34:18.672147 17984 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:34:18.674793 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.656417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45618 (local address 127.4.83.124:37241)
0619 20:34:18.656545 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:18.656546 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:18.668074 (+ 11528us) server_negotiation.cc:407] Connection header received
0619 20:34:18.668169 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:18.668170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:18.668182 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:18.668227 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:18.668810 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.669357 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.669987 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.670199 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.670253 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:18.671327 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:18.671328 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:18.671329 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:18.671336 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:18.673112 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:18.673505 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:18.673507 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:18.673507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:18.673569 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:18.673760 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:18.673761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:18.673762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:18.673910 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:18.674011 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:18.674240 (+ 229us) server_negotiation.cc:299] Negotiation successful
0619 20:34:18.674337 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
I20260619 20:34:18.697189 17879 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.83.124" port: 37241 } attempt: 3
Jun 19 20:34:18 dist-test-slave-4m58 krb5kdc[17722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901258, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.125@KRBTEST.COM for kudu/127.4.83.124@KRBTEST.COM
I20260619 20:34:18.706908 17881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.697572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:37241 (local address 127.0.0.1:45620)
0619 20:34:18.697637 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:18.697649 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:18.697687 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:18.697850 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:18.697851 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:18.698056 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:34:18.698115 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:18.698117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.699131 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.699132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:18.700014 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:18.700018 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.700526 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.700528 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:18.700590 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:18.701393 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:34:18.701399 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:34:18.703631 (+ 2232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:34:18.705729 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:18.705730 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:18.705731 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:18.705978 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:18.706182 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:18.706183 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:18.706183 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:18.706249 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:18.706528 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:34:18.706529 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:34:18.706613 (+ 84us) client_negotiation.cc:769] Sending connection context
0619 20:34:18.706684 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 20:34:18.706797 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 20:34:18.706981 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:18.697639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45620 (local address 127.4.83.124:37241)
0619 20:34:18.697686 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:18.697688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:18.697699 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:34:18.697754 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:18.697755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:18.697767 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:18.697838 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:18.698191 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.699021 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.700112 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:18.700439 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:18.700781 (+ 342us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:18.703780 (+ 2999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:18.703782 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:18.703782 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:18.703790 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:18.705637 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:18.706062 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:18.706063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:18.706063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:18.706123 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:18.706306 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:18.706307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:18.706307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:18.706464 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:18.706528 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:18.706752 (+ 224us) server_negotiation.cc:299] Negotiation successful
0619 20:34:18.706878 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:34:18.709218 17879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0129451a23974a11832e01fbaed0a60f: Bootstrap starting.
I20260619 20:34:18.710008 17879 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0129451a23974a11832e01fbaed0a60f: Neither blocks nor log segments found. Creating new log.
I20260619 20:34:18.710342 17879 log.cc:824] T 00000000000000000000000000000000 P 0129451a23974a11832e01fbaed0a60f: Log is configured to *not* fsync() on all Append() calls
I20260619 20:34:18.711717 17879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0129451a23974a11832e01fbaed0a60f: No bootstrap required, opened a new log
I20260619 20:34:18.713687 17984 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:34:18.713809 17879 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0129451a23974a11832e01fbaed0a60f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 32917 } } peers { permanent_uuid: "0129451a23974a11832e01fbaed0a60f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 43911 } } peers { permanent_uuid: "2e4d47fc9de94dabb94915678b001d54" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 37241 } }
I20260619 20:34:18.713855 17984 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 32917 } } peers { permanent_uuid: "0129451a23974a11832e01fbaed0a60f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 43911 } } peers { permanent_uuid: "2e4d47fc9de94dabb94915678b001d54" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 37241 } }
I20260619 20:34:18.714020 17879 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0129451a23974a11832e01fbaed0a60f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:34:18.714092 17879 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0129451a23974a11832e01fbaed0a60f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0129451a23974a11832e01fbaed0a60f, State: Initialized, Role: FOLLOWER
I20260619 20:34:18.714250 17879 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0129451a23974a11832e01fbaed0a60f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 32917 } } peers { permanent_uuid: "0129451a23974a11832e01fbaed0a60f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 43911 } } peers { permanent_uuid: "2e4d47fc9de94dabb94915678b001d54" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 37241 } }
I20260619 20:34:18.714684 17879 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0129451a23974a11832e01fbaed0a60f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:34:18.714854 17928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e4d47fc9de94dabb94915678b001d54" is_pre_election: true
I20260619 20:34:18.715026 17928 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2e4d47fc9de94dabb94915678b001d54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0cd06fe4bc94a5a9dd50132cbbcee0b in term 0.
I20260619 20:34:18.715206 17853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0129451a23974a11832e01fbaed0a60f" is_pre_election: true
I20260619 20:34:18.715318 17853 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0129451a23974a11832e01fbaed0a60f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0cd06fe4bc94a5a9dd50132cbbcee0b in term 0.
I20260619 20:34:18.715663 17754 leader_election.cc:302] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0129451a23974a11832e01fbaed0a60f, a0cd06fe4bc94a5a9dd50132cbbcee0b; no voters:
I20260619 20:34:18.716130 17984 leader_election.cc:288] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0129451a23974a11832e01fbaed0a60f (127.4.83.125:43911), 2e4d47fc9de94dabb94915678b001d54 (127.4.83.124:37241)
I20260619 20:34:18.716327 17984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:34:18.716418 17984 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:34:18.716485 17984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b [term 0 FOLLOWER]: Advancing to term 1
W20260619 20:34:18.716943 18010 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0129451a23974a11832e01fbaed0a60f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:34:18.717054 18010 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:34:18.717460 17984 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 32917 } } peers { permanent_uuid: "0129451a23974a11832e01fbaed0a60f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 43911 } } peers { permanent_uuid: "2e4d47fc9de94dabb94915678b001d54" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 37241 } }
I20260619 20:34:18.717816 17853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0129451a23974a11832e01fbaed0a60f"
I20260619 20:34:18.717898 17853 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0129451a23974a11832e01fbaed0a60f [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:34:18.718720 17853 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0129451a23974a11832e01fbaed0a60f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0cd06fe4bc94a5a9dd50132cbbcee0b in term 1.
I20260619 20:34:18.718930 17754 leader_election.cc:302] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0129451a23974a11832e01fbaed0a60f, a0cd06fe4bc94a5a9dd50132cbbcee0b; no voters:
I20260619 20:34:18.719204 17928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e4d47fc9de94dabb94915678b001d54"
I20260619 20:34:18.719287 17928 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e4d47fc9de94dabb94915678b001d54 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:34:18.719928 18007 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:34:18.720088 18007 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b [term 1 LEADER]: Becoming Leader. State: Replica: a0cd06fe4bc94a5a9dd50132cbbcee0b, State: Running, Role: LEADER
I20260619 20:34:18.720105 17928 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2e4d47fc9de94dabb94915678b001d54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0cd06fe4bc94a5a9dd50132cbbcee0b in term 1.
I20260619 20:34:18.720171 18007 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 32917 } } peers { permanent_uuid: "0129451a23974a11832e01fbaed0a60f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 43911 } } peers { permanent_uuid: "2e4d47fc9de94dabb94915678b001d54" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 37241 } }
I20260619 20:34:18.720570 18007 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0cd06fe4bc94a5a9dd50132cbbcee0b. Latest consensus state: current_term: 1 leader_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 32917 } } peers { permanent_uuid: "0129451a23974a11832e01fbaed0a60f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 43911 } } peers { permanent_uuid: "2e4d47fc9de94dabb94915678b001d54" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 37241 } } }
I20260619 20:34:18.720705 18007 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b [sys.catalog]: This master's current role is: LEADER
I20260619 20:34:18.721086 17998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0129451a23974a11832e01fbaed0a60f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 32917 } } peers { permanent_uuid: "0129451a23974a11832e01fbaed0a60f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 43911 } } peers { permanent_uuid: "2e4d47fc9de94dabb94915678b001d54" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 37241 } } }
I20260619 20:34:18.721226 17998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0129451a23974a11832e01fbaed0a60f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:34:18.721375 17984 leader_election.cc:288] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b [CANDIDATE]: Term 1 election: Requested vote from peers 0129451a23974a11832e01fbaed0a60f (127.4.83.125:43911), 2e4d47fc9de94dabb94915678b001d54 (127.4.83.124:37241)
I20260619 20:34:18.721408 17811 master_runner.cc:263] No leader master found from master 0129451a23974a11832e01fbaed0a60f
I20260619 20:34:18.721438 17811 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 20:34:18.728058 18014 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:34:18.728291 18014 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:34:18.732435 17853 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0129451a23974a11832e01fbaed0a60f [term 1 FOLLOWER]: Refusing update from remote peer a0cd06fe4bc94a5a9dd50132cbbcee0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:34:18.733392 17928 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2e4d47fc9de94dabb94915678b001d54 [term 1 FOLLOWER]: Refusing update from remote peer a0cd06fe4bc94a5a9dd50132cbbcee0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:34:18.734102 17984 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e4d47fc9de94dabb94915678b001d54" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 37241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:34:18.735966 17961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e4d47fc9de94dabb94915678b001d54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0cd06fe4bc94a5a9dd50132cbbcee0b. Latest consensus state: current_term: 1 leader_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 32917 } } peers { permanent_uuid: "0129451a23974a11832e01fbaed0a60f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 43911 } } peers { permanent_uuid: "2e4d47fc9de94dabb94915678b001d54" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 37241 } } }
I20260619 20:34:18.736059 17961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e4d47fc9de94dabb94915678b001d54 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:34:18.736207 17984 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0129451a23974a11832e01fbaed0a60f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 43911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:34:18.737481 17998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0129451a23974a11832e01fbaed0a60f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0cd06fe4bc94a5a9dd50132cbbcee0b. Latest consensus state: current_term: 1 leader_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 32917 } } peers { permanent_uuid: "0129451a23974a11832e01fbaed0a60f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 43911 } } peers { permanent_uuid: "2e4d47fc9de94dabb94915678b001d54" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 37241 } } }
I20260619 20:34:18.737586 17998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0129451a23974a11832e01fbaed0a60f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:34:18.739063 17984 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 32917 } } peers { permanent_uuid: "0129451a23974a11832e01fbaed0a60f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 43911 } } peers { permanent_uuid: "2e4d47fc9de94dabb94915678b001d54" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 37241 } } }
I20260619 20:34:18.739156 17984 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b [sys.catalog]: This master's current role is: LEADER
I20260619 20:34:18.739693 17961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e4d47fc9de94dabb94915678b001d54 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 32917 } } peers { permanent_uuid: "0129451a23974a11832e01fbaed0a60f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 43911 } } peers { permanent_uuid: "2e4d47fc9de94dabb94915678b001d54" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 37241 } } }
I20260619 20:34:18.739784 17961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e4d47fc9de94dabb94915678b001d54 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:34:18.740211 18015 mvcc.cc:205] Tried to move back new op lower bound from 7298667555763527680 to 7298667555718471680. Current Snapshot: MvccSnapshot[applied={T|T < 7298667555763527680}]
I20260619 20:34:18.740444 18016 mvcc.cc:205] Tried to move back new op lower bound from 7298667555763527680 to 7298667555718471680. Current Snapshot: MvccSnapshot[applied={T|T < 7298667555763527680}]
I20260619 20:34:18.741839 18007 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 32917 } } peers { permanent_uuid: "0129451a23974a11832e01fbaed0a60f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 43911 } } peers { permanent_uuid: "2e4d47fc9de94dabb94915678b001d54" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 37241 } } }
I20260619 20:34:18.741925 18007 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b [sys.catalog]: This master's current role is: LEADER
I20260619 20:34:18.742591 17998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0129451a23974a11832e01fbaed0a60f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd06fe4bc94a5a9dd50132cbbcee0b" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 32917 } } peers { permanent_uuid: "0129451a23974a11832e01fbaed0a60f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 43911 } } peers { permanent_uuid: "2e4d47fc9de94dabb94915678b001d54" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 37241 } } }
I20260619 20:34:18.742682 17998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0129451a23974a11832e01fbaed0a60f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:34:18.742985 18018 mvcc.cc:205] Tried to move back new op lower bound from 7298667555763527680 to 7298667555718471680. Current Snapshot: MvccSnapshot[applied={T|T < 7298667555763527680}]
I20260619 20:34:18.744035 18014 catalog_manager.cc:1347] Generated new cluster ID: f51983db65834ba9ac3e64efbe7476ef
I20260619 20:34:18.744082 18014 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:34:18.754747 18014 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:34:18.755867 18014 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:34:18.761898 18014 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0cd06fe4bc94a5a9dd50132cbbcee0b: Generated new TSK 0
I20260619 20:34:18.762166 18014 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:34:19.656634 17974 catalog_manager.cc:1259] Loaded cluster ID: f51983db65834ba9ac3e64efbe7476ef
I20260619 20:34:19.656714 17974 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2e4d47fc9de94dabb94915678b001d54: loading cluster ID for follower catalog manager: success
I20260619 20:34:19.657549 17974 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2e4d47fc9de94dabb94915678b001d54: acquiring CA information for follower catalog manager: success
I20260619 20:34:19.657846 17974 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2e4d47fc9de94dabb94915678b001d54: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 20:34:19.717537 18010 catalog_manager.cc:1259] Loaded cluster ID: f51983db65834ba9ac3e64efbe7476ef
I20260619 20:34:19.717618 18010 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0129451a23974a11832e01fbaed0a60f: loading cluster ID for follower catalog manager: success
I20260619 20:34:19.718436 18010 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0129451a23974a11832e01fbaed0a60f: acquiring CA information for follower catalog manager: success
I20260619 20:34:19.718734 18010 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0129451a23974a11832e01fbaed0a60f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 20:34:19.814224 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 17735
I20260619 20:34:19.820013 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 17811
I20260619 20:34:19.825314 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 17886
2026-06-19T20:34:19Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2744 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:34:19 dist-test-slave-4m58 krb5kdc[18031](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:34:19 dist-test-slave-4m58 krb5kdc[18031](info): set up 2 sockets
Jun 19 20:34:19 dist-test-slave-4m58 krb5kdc[18031](info): commencing operation
krb5kdc: starting...
W20260619 20:34:20.800915 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.954s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:34:20 dist-test-slave-4m58 krb5kdc[18031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:34:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:34:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
I20260619 20:34:20.929401 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:39647
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:40371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:39647
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:34:21.009122 18045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:34:21.009375 18045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:34:21.009411 18045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:34:21.010679 18045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:34:21.010731 18045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:34:21.010751 18045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:34:21.010766 18045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:34:21.010782 18045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:34:21.012339 18045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:40371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:39647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:39647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:34:21.012602 18045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:34:21.012825 18045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:34:21.018108 18050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:21.018114 18051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:21.018108 18053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:21.018302 18045 server_base.cc:1034] running on GCE node
I20260619 20:34:21.018682 18045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:34:21.019001 18045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:34:21.020151 18045 hybrid_clock.cc:648] HybridClock initialized: now 1781901261020130 us; error 48 us; skew 500 ppm
Jun 19 20:34:21 dist-test-slave-4m58 krb5kdc[18031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901261, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:34:21.022907 18045 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:34:21.023432 18045 webserver.cc:460] Webserver started at http://127.4.83.126:45009/ using document root <none> and password file <none>
I20260619 20:34:21.023671 18045 fs_manager.cc:359] Metadata directory not provided
I20260619 20:34:21.023723 18045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:34:21.023896 18045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:34:21.025015 18045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "99efee4cd0174f17b9ed7e2755730170"
format_stamp: "Formatted at 2026-06-19 20:34:21 on dist-test-slave-4m58"
server_key: "e47305a93fe82e314c706311ed153e80"
server_key_iv: "fb5eac4b3523ea8b001a93508c841189"
server_key_version: "encryptionkey@0"
I20260619 20:34:21.025377 18045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "99efee4cd0174f17b9ed7e2755730170"
format_stamp: "Formatted at 2026-06-19 20:34:21 on dist-test-slave-4m58"
server_key: "e47305a93fe82e314c706311ed153e80"
server_key_iv: "fb5eac4b3523ea8b001a93508c841189"
server_key_version: "encryptionkey@0"
I20260619 20:34:21.026834 18045 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:34:21.027649 18060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:21.027882 18045 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:34:21.027987 18045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "99efee4cd0174f17b9ed7e2755730170"
format_stamp: "Formatted at 2026-06-19 20:34:21 on dist-test-slave-4m58"
server_key: "e47305a93fe82e314c706311ed153e80"
server_key_iv: "fb5eac4b3523ea8b001a93508c841189"
server_key_version: "encryptionkey@0"
I20260619 20:34:21.028061 18045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:34:21.041718 18045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:34:21.043941 18045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:34:21.044085 18045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:34:21.048388 18045 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:39647
I20260619 20:34:21.048460 18112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:39647 every 8 connection(s)
I20260619 20:34:21.048825 18045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:34:21.049916 18114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:34:21.053194 18114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99efee4cd0174f17b9ed7e2755730170: Bootstrap starting.
I20260619 20:34:21.053990 18114 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99efee4cd0174f17b9ed7e2755730170: Neither blocks nor log segments found. Creating new log.
I20260619 20:34:21.054281 18114 log.cc:824] T 00000000000000000000000000000000 P 99efee4cd0174f17b9ed7e2755730170: Log is configured to *not* fsync() on all Append() calls
I20260619 20:34:21.055290 18114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99efee4cd0174f17b9ed7e2755730170: No bootstrap required, opened a new log
I20260619 20:34:21.055312 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 18045
I20260619 20:34:21.055428 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:34:21.055620 4429 external_mini_cluster.cc:1428] Setting key ce592f8315c2041b665a493bc73f14aa
I20260619 20:34:21.057847 18114 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99efee4cd0174f17b9ed7e2755730170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99efee4cd0174f17b9ed7e2755730170" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 39647 } }
I20260619 20:34:21.058102 18114 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99efee4cd0174f17b9ed7e2755730170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:34:21.058142 18114 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99efee4cd0174f17b9ed7e2755730170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99efee4cd0174f17b9ed7e2755730170, State: Initialized, Role: FOLLOWER
I20260619 20:34:21.058367 18114 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99efee4cd0174f17b9ed7e2755730170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99efee4cd0174f17b9ed7e2755730170" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 39647 } }
I20260619 20:34:21.058463 18114 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99efee4cd0174f17b9ed7e2755730170 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:34:21.058490 18114 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99efee4cd0174f17b9ed7e2755730170 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:34:21.058528 18114 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99efee4cd0174f17b9ed7e2755730170 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:34:21.059563 18114 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99efee4cd0174f17b9ed7e2755730170 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99efee4cd0174f17b9ed7e2755730170" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 39647 } }
I20260619 20:34:21.059712 18114 leader_election.cc:302] T 00000000000000000000000000000000 P 99efee4cd0174f17b9ed7e2755730170 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99efee4cd0174f17b9ed7e2755730170; no voters:
I20260619 20:34:21.059963 18114 leader_election.cc:288] T 00000000000000000000000000000000 P 99efee4cd0174f17b9ed7e2755730170 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:34:21.060050 18119 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99efee4cd0174f17b9ed7e2755730170 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:34:21.060325 18119 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99efee4cd0174f17b9ed7e2755730170 [term 1 LEADER]: Becoming Leader. State: Replica: 99efee4cd0174f17b9ed7e2755730170, State: Running, Role: LEADER
I20260619 20:34:21.060309 18114 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99efee4cd0174f17b9ed7e2755730170 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:34:21.060541 18119 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99efee4cd0174f17b9ed7e2755730170 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99efee4cd0174f17b9ed7e2755730170" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 39647 } }
I20260619 20:34:21.061013 18120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99efee4cd0174f17b9ed7e2755730170 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99efee4cd0174f17b9ed7e2755730170" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99efee4cd0174f17b9ed7e2755730170" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 39647 } } }
I20260619 20:34:21.061106 18120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99efee4cd0174f17b9ed7e2755730170 [sys.catalog]: This master's current role is: LEADER
I20260619 20:34:21.061367 18121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99efee4cd0174f17b9ed7e2755730170 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99efee4cd0174f17b9ed7e2755730170. Latest consensus state: current_term: 1 leader_uuid: "99efee4cd0174f17b9ed7e2755730170" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99efee4cd0174f17b9ed7e2755730170" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 39647 } } }
I20260619 20:34:21.061488 18121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99efee4cd0174f17b9ed7e2755730170 [sys.catalog]: This master's current role is: LEADER
I20260619 20:34:21.061756 18127 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:34:21.062479 18127 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:34:21.064678 18127 catalog_manager.cc:1347] Generated new cluster ID: adf47cb2b0cf4ba595eaafa5fce7eb82
I20260619 20:34:21.064755 18127 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:34:21 dist-test-slave-4m58 krb5kdc[18031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:34:21.069522 18118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:21.056281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34424 (local address 127.4.83.126:39647)
0619 20:34:21.056569 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:21.056582 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:21.056624 (+ 42us) server_negotiation.cc:407] Connection header received
0619 20:34:21.057770 (+ 1146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:21.057782 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:21.057876 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:21.058089 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:21.058888 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:21.060714 (+ 1826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:21.061906 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:21.062542 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:21.062659 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:21.065791 (+ 3132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:21.065798 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:21.065798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:21.065819 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:21.067668 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:21.068145 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:21.068149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:21.068150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:21.068275 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:21.068473 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:21.068474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:21.068474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:21.068802 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:21.069002 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:21.069084 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 20:34:21.069272 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":133,"threads_started":1}
I20260619 20:34:21.070590 4429 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:34:21.074442 18127 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:34:21.075209 18127 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:34:21.079718 18127 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99efee4cd0174f17b9ed7e2755730170: Generated new TSK 0
I20260619 20:34:21.079972 18127 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:34:21.125159 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 18045
2026-06-19T20:34:21Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1297 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:34:21 dist-test-slave-4m58 krb5kdc[18141](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:34:21 dist-test-slave-4m58 krb5kdc[18141](info): set up 2 sockets
Jun 19 20:34:21 dist-test-slave-4m58 krb5kdc[18141](info): commencing operation
krb5kdc: starting...
W20260619 20:34:22.104799 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.961s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:34:22 dist-test-slave-4m58 krb5kdc[18141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:34:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:34:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
I20260619 20:34:22.237154 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:41283
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:33145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:41283
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:34:22.320379 18154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:34:22.320611 18154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:34:22.320640 18154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:34:22.321887 18154 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:34:22.321940 18154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:34:22.321959 18154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:34:22.321974 18154 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:34:22.321988 18154 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:34:22.323575 18154 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:33145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:41283
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:41283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:34:22.323887 18154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:34:22.324112 18154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:34:22.329419 18160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:22.329424 18162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:22.329438 18159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:22.329607 18154 server_base.cc:1034] running on GCE node
I20260619 20:34:22.329943 18154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:34:22.330269 18154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:34:22.331432 18154 hybrid_clock.cc:648] HybridClock initialized: now 1781901262331413 us; error 38 us; skew 500 ppm
Jun 19 20:34:22 dist-test-slave-4m58 krb5kdc[18141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901262, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:34:22.334157 18154 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:34:22.334645 18154 webserver.cc:460] Webserver started at http://127.4.83.126:35161/ using document root <none> and password file <none>
I20260619 20:34:22.334867 18154 fs_manager.cc:359] Metadata directory not provided
I20260619 20:34:22.334915 18154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:34:22.335045 18154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:34:22.336139 18154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "f6176e2299aa4187ad26ef24af4f4ba3"
format_stamp: "Formatted at 2026-06-19 20:34:22 on dist-test-slave-4m58"
server_key: "9b80028812366155afb87c71b4374df3"
server_key_iv: "51387129ce274ef1c1cb26144ac8aa9c"
server_key_version: "encryptionkey@0"
I20260619 20:34:22.336460 18154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "f6176e2299aa4187ad26ef24af4f4ba3"
format_stamp: "Formatted at 2026-06-19 20:34:22 on dist-test-slave-4m58"
server_key: "9b80028812366155afb87c71b4374df3"
server_key_iv: "51387129ce274ef1c1cb26144ac8aa9c"
server_key_version: "encryptionkey@0"
I20260619 20:34:22.337831 18154 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:34:22.338680 18169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:22.339015 18154 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:34:22.339129 18154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "f6176e2299aa4187ad26ef24af4f4ba3"
format_stamp: "Formatted at 2026-06-19 20:34:22 on dist-test-slave-4m58"
server_key: "9b80028812366155afb87c71b4374df3"
server_key_iv: "51387129ce274ef1c1cb26144ac8aa9c"
server_key_version: "encryptionkey@0"
I20260619 20:34:22.339205 18154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:34:22.347060 18154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:34:22.349287 18154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:34:22.349431 18154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:34:22.353708 18154 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:41283
I20260619 20:34:22.353842 18221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:41283 every 8 connection(s)
I20260619 20:34:22.354115 18154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:34:22.355283 18222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:34:22.358273 18222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3: Bootstrap starting.
I20260619 20:34:22.358944 18222 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3: Neither blocks nor log segments found. Creating new log.
I20260619 20:34:22.359210 18222 log.cc:824] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3: Log is configured to *not* fsync() on all Append() calls
I20260619 20:34:22.360155 18222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3: No bootstrap required, opened a new log
I20260619 20:34:22.361936 18222 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6176e2299aa4187ad26ef24af4f4ba3" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 41283 } }
I20260619 20:34:22.362105 18222 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:34:22.362128 18222 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6176e2299aa4187ad26ef24af4f4ba3, State: Initialized, Role: FOLLOWER
I20260619 20:34:22.362262 18222 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6176e2299aa4187ad26ef24af4f4ba3" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 41283 } }
I20260619 20:34:22.362325 18222 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:34:22.362347 18222 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:34:22.362375 18222 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:34:22.363125 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 18154
I20260619 20:34:22.363248 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:34:22.363355 18222 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6176e2299aa4187ad26ef24af4f4ba3" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 41283 } }
I20260619 20:34:22.363464 4429 external_mini_cluster.cc:1428] Setting key b1aa28a2381c4b7f8592565b9e1d67d9
I20260619 20:34:22.363469 18222 leader_election.cc:302] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6176e2299aa4187ad26ef24af4f4ba3; no voters:
I20260619 20:34:22.363662 18222 leader_election.cc:288] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:34:22.363749 18225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:34:22.364022 18222 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:34:22.364096 18225 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [term 1 LEADER]: Becoming Leader. State: Replica: f6176e2299aa4187ad26ef24af4f4ba3, State: Running, Role: LEADER
I20260619 20:34:22.364272 18225 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6176e2299aa4187ad26ef24af4f4ba3" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 41283 } }
I20260619 20:34:22.364943 18228 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6176e2299aa4187ad26ef24af4f4ba3. Latest consensus state: current_term: 1 leader_uuid: "f6176e2299aa4187ad26ef24af4f4ba3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6176e2299aa4187ad26ef24af4f4ba3" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 41283 } } }
I20260619 20:34:22.365046 18228 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [sys.catalog]: This master's current role is: LEADER
I20260619 20:34:22.365098 18226 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6176e2299aa4187ad26ef24af4f4ba3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6176e2299aa4187ad26ef24af4f4ba3" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 41283 } } }
I20260619 20:34:22.365151 18226 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [sys.catalog]: This master's current role is: LEADER
I20260619 20:34:22.365717 18236 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:34:22.366425 18236 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:34:22.369119 18236 catalog_manager.cc:1347] Generated new cluster ID: ecdff7059dd640ddaa527d1be8940d7d
I20260619 20:34:22.369254 18236 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:34:22 dist-test-slave-4m58 krb5kdc[18141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:34:22.378036 18232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:22.364696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44046 (local address 127.4.83.126:41283)
0619 20:34:22.365263 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:22.365275 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:22.365324 (+ 49us) server_negotiation.cc:407] Connection header received
0619 20:34:22.365480 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:22.365489 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:22.365554 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:22.366606 (+ 1052us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:22.366741 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:22.368184 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:22.369047 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:22.369673 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:22.369782 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:22.373444 (+ 3662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:22.373458 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:22.373459 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:22.373482 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:22.375972 (+ 2490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:22.376457 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:22.376462 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:22.376463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:22.376632 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:22.376842 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:22.376843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:22.376844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:22.377259 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:22.377477 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:22.377558 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 20:34:22.377747 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":91,"threads_started":1}
I20260619 20:34:22.383543 18236 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:34:22.384225 18236 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:34:22.388144 18236 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3: Generated new TSK 0
I20260619 20:34:22.388335 18236 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.83.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
I20260619 20:34:22.418848 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:0
--local_ip_for_outbound_sockets=127.4.83.65
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:41283
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:33145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:34:22.505446 18248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:34:22.505692 18248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:34:22.505720 18248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:34:22.507026 18248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:34:22.507090 18248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:34:22.507129 18248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:34:22.508831 18248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:33145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:41283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:34:22.509097 18248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:34:22.509363 18248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:34:22.514945 18257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:22.514945 18254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:22.514921 18255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:22.515262 18248 server_base.cc:1034] running on GCE node
I20260619 20:34:22.515535 18248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:34:22.515872 18248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:34:22.517042 18248 hybrid_clock.cc:648] HybridClock initialized: now 1781901262517032 us; error 32 us; skew 500 ppm
Jun 19 20:34:22 dist-test-slave-4m58 krb5kdc[18141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901262, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:34:22.519954 18248 init.cc:377] Logged in from keytab as kudu/127.4.83.65@KRBTEST.COM (short username kudu)
I20260619 20:34:22.520488 18248 webserver.cc:460] Webserver started at http://127.4.83.65:40069/ using document root <none> and password file <none>
I20260619 20:34:22.520753 18248 fs_manager.cc:359] Metadata directory not provided
I20260619 20:34:22.520805 18248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:34:22.520944 18248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:34:22.522197 18248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/data/instance:
uuid: "307138c79bd84bb5b8d3791eecfe5940"
format_stamp: "Formatted at 2026-06-19 20:34:22 on dist-test-slave-4m58"
server_key: "d775174f80afc71a3ce59d737b51cfd8"
server_key_iv: "e69b93b97fb23c33350661ac3b91cee2"
server_key_version: "encryptionkey@0"
I20260619 20:34:22.522558 18248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance:
uuid: "307138c79bd84bb5b8d3791eecfe5940"
format_stamp: "Formatted at 2026-06-19 20:34:22 on dist-test-slave-4m58"
server_key: "d775174f80afc71a3ce59d737b51cfd8"
server_key_iv: "e69b93b97fb23c33350661ac3b91cee2"
server_key_version: "encryptionkey@0"
I20260619 20:34:22.524291 18248 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260619 20:34:22.525415 18264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:22.525687 18248 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:34:22.525780 18248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "307138c79bd84bb5b8d3791eecfe5940"
format_stamp: "Formatted at 2026-06-19 20:34:22 on dist-test-slave-4m58"
server_key: "d775174f80afc71a3ce59d737b51cfd8"
server_key_iv: "e69b93b97fb23c33350661ac3b91cee2"
server_key_version: "encryptionkey@0"
I20260619 20:34:22.525862 18248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:34:22.546104 18248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:34:22.548501 18248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:34:22.548640 18248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:34:22.548897 18248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:34:22.549252 18248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:34:22.549288 18248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:22.549314 18248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:34:22.549341 18248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:22.556704 18248 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:44361
I20260619 20:34:22.556808 18377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:44361 every 8 connection(s)
I20260619 20:34:22.557190 18248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
I20260619 20:34:22.564355 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 18248
I20260619 20:34:22.564494 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance
Jun 19 20:34:22 dist-test-slave-4m58 krb5kdc[18141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901262, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:34:22.564761 4429 external_mini_cluster.cc:1428] Setting key fd5f3d65aa85ed3016cfb759517be5f2
I20260619 20:34:22.569561 18232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:22.558132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:42243 (local address 127.4.83.126:41283)
0619 20:34:22.558229 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:22.558231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:22.558448 (+ 217us) server_negotiation.cc:407] Connection header received
0619 20:34:22.559062 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:22.559064 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:22.559076 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:22.559160 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:22.560371 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:22.560942 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:22.562079 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:22.562307 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:22.562384 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:22.565507 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:22.565509 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:22.565510 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:22.565518 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:22.567164 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:22.567986 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:22.567987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:22.567987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:22.568063 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:22.568435 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:22.568436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:22.568437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:22.568591 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:22.568659 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:22.569358 (+ 699us) server_negotiation.cc:299] Negotiation successful
0619 20:34:22.569469 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 20:34:22.569764 18380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:22.558071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.65:42243)
0619 20:34:22.558270 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:22.558326 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:22.558938 (+ 612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:22.559243 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:22.559247 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:22.559976 (+ 729us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:34:22.560227 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:22.560232 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:22.561040 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:22.561040 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:22.561964 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:22.561967 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:22.562440 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:22.562441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:22.562582 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:22.563230 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:34:22.563240 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:34:22.565391 (+ 2151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:34:22.567383 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:22.567398 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:22.567402 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:22.567882 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:22.568163 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:22.568164 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:22.568164 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:22.568312 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:22.568672 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:34:22.568679 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:34:22.569100 (+ 421us) client_negotiation.cc:769] Sending connection context
0619 20:34:22.569274 (+ 174us) client_negotiation.cc:240] Negotiation successful
0619 20:34:22.569503 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":109,"threads_started":1}
I20260619 20:34:22.570559 18378 heartbeater.cc:347] Connected to a master server at 127.4.83.126:41283
I20260619 20:34:22.570686 18378 heartbeater.cc:464] Registering TS with master...
I20260619 20:34:22.570937 18378 heartbeater.cc:511] Master 127.4.83.126:41283 requested a full tablet report, sending...
I20260619 20:34:22.571677 18186 ts_manager.cc:194] Registered new tserver with Master: 307138c79bd84bb5b8d3791eecfe5940 (127.4.83.65:44361)
I20260619 20:34:22.572460 18186 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.65@KRBTEST.COM'} at 127.4.83.65:42243
WARNING: no policy specified for kudu/127.4.83.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
I20260619 20:34:22.594625 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:0
--local_ip_for_outbound_sockets=127.4.83.66
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:41283
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:33145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:34:22.680999 18383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:34:22.681233 18383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:34:22.681268 18383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:34:22.682607 18383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:34:22.682669 18383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:34:22.682710 18383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:34:22.684568 18383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:33145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:41283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:34:22.684854 18383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:34:22.685094 18383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:34:22.690819 18389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:22.690835 18391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:22.690816 18388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:22.691185 18383 server_base.cc:1034] running on GCE node
I20260619 20:34:22.691403 18383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:34:22.691653 18383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:34:22.692826 18383 hybrid_clock.cc:648] HybridClock initialized: now 1781901262692796 us; error 48 us; skew 500 ppm
Jun 19 20:34:22 dist-test-slave-4m58 krb5kdc[18141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901262, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:34:22.695976 18383 init.cc:377] Logged in from keytab as kudu/127.4.83.66@KRBTEST.COM (short username kudu)
I20260619 20:34:22.696491 18383 webserver.cc:460] Webserver started at http://127.4.83.66:37767/ using document root <none> and password file <none>
I20260619 20:34:22.696718 18383 fs_manager.cc:359] Metadata directory not provided
I20260619 20:34:22.696771 18383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:34:22.696907 18383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:34:22.698164 18383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/data/instance:
uuid: "0e4f581737d04471b15ec0dbcdd5a5fa"
format_stamp: "Formatted at 2026-06-19 20:34:22 on dist-test-slave-4m58"
server_key: "c0c3b6184e7bf83659f4f26efaee3c20"
server_key_iv: "f9c03385c9f44da91e936158f8bdce69"
server_key_version: "encryptionkey@0"
I20260619 20:34:22.698539 18383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance:
uuid: "0e4f581737d04471b15ec0dbcdd5a5fa"
format_stamp: "Formatted at 2026-06-19 20:34:22 on dist-test-slave-4m58"
server_key: "c0c3b6184e7bf83659f4f26efaee3c20"
server_key_iv: "f9c03385c9f44da91e936158f8bdce69"
server_key_version: "encryptionkey@0"
I20260619 20:34:22.700058 18383 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:34:22.700991 18399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:22.701251 18383 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:34:22.701351 18383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "0e4f581737d04471b15ec0dbcdd5a5fa"
format_stamp: "Formatted at 2026-06-19 20:34:22 on dist-test-slave-4m58"
server_key: "c0c3b6184e7bf83659f4f26efaee3c20"
server_key_iv: "f9c03385c9f44da91e936158f8bdce69"
server_key_version: "encryptionkey@0"
I20260619 20:34:22.701447 18383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:34:22.731735 18383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:34:22.734170 18383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:34:22.734330 18383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:34:22.734578 18383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:34:22.734898 18383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:34:22.734938 18383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:22.734969 18383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:34:22.734985 18383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:22.741557 18383 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:37693
I20260619 20:34:22.741711 18512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:37693 every 8 connection(s)
I20260619 20:34:22.741989 18383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
Jun 19 20:34:22 dist-test-slave-4m58 krb5kdc[18141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901262, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:34:22.751322 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 18383
I20260619 20:34:22.751441 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance
I20260619 20:34:22.751639 4429 external_mini_cluster.cc:1428] Setting key eae99c326451d21c73ded844d0c4160a
I20260619 20:34:22.753077 18232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:22.742744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:47269 (local address 127.4.83.126:41283)
0619 20:34:22.742817 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:22.742819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:22.743088 (+ 269us) server_negotiation.cc:407] Connection header received
0619 20:34:22.743788 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:22.743789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:22.743804 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:22.743878 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:22.745040 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:22.745602 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:22.746707 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:22.746917 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:22.746980 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:22.749638 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:22.749640 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:22.749641 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:22.749652 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:22.751067 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:22.751794 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:22.751797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:22.751797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:22.751961 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:22.752215 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:22.752216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:22.752217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:22.752355 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:22.752436 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:22.752856 (+ 420us) server_negotiation.cc:299] Negotiation successful
0619 20:34:22.753001 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:34:22.753353 18515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:22.742716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.66:47269)
0619 20:34:22.742913 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:22.742967 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:22.743664 (+ 697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:22.743966 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:22.743972 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:22.744666 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:34:22.744922 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:22.744928 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:22.745703 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:22.745704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:22.746607 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:22.746610 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:22.746997 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:22.746998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:22.747143 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:22.747795 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:34:22.747806 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:34:22.749509 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:34:22.751237 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:22.751253 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:22.751255 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:22.751676 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:22.752021 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:22.752022 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:22.752022 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:22.752141 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:22.752428 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:34:22.752436 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:34:22.752658 (+ 222us) client_negotiation.cc:769] Sending connection context
0619 20:34:22.752810 (+ 152us) client_negotiation.cc:240] Negotiation successful
0619 20:34:22.753047 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":98,"threads_started":1}
I20260619 20:34:22.754004 18513 heartbeater.cc:347] Connected to a master server at 127.4.83.126:41283
I20260619 20:34:22.754117 18513 heartbeater.cc:464] Registering TS with master...
I20260619 20:34:22.754315 18513 heartbeater.cc:511] Master 127.4.83.126:41283 requested a full tablet report, sending...
I20260619 20:34:22.754717 18186 ts_manager.cc:194] Registered new tserver with Master: 0e4f581737d04471b15ec0dbcdd5a5fa (127.4.83.66:37693)
I20260619 20:34:22.755111 18186 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.66@KRBTEST.COM'} at 127.4.83.66:47269
WARNING: no policy specified for kudu/127.4.83.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
I20260619 20:34:22.779337 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:0
--local_ip_for_outbound_sockets=127.4.83.67
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:41283
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:33145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:34:22.862958 18518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:34:22.863195 18518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:34:22.863230 18518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:34:22.864574 18518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:34:22.864627 18518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:34:22.864668 18518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:34:22.866349 18518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:33145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:41283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:34:22.866618 18518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:34:22.866854 18518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:34:22.872347 18523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:22.872398 18524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:22.872407 18526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:22.872553 18518 server_base.cc:1034] running on GCE node
I20260619 20:34:22.872987 18518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:34:22.873327 18518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:34:22.874521 18518 hybrid_clock.cc:648] HybridClock initialized: now 1781901262874482 us; error 60 us; skew 500 ppm
Jun 19 20:34:22 dist-test-slave-4m58 krb5kdc[18141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901262, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:34:22.877993 18518 init.cc:377] Logged in from keytab as kudu/127.4.83.67@KRBTEST.COM (short username kudu)
I20260619 20:34:22.878602 18518 webserver.cc:460] Webserver started at http://127.4.83.67:39573/ using document root <none> and password file <none>
I20260619 20:34:22.878881 18518 fs_manager.cc:359] Metadata directory not provided
I20260619 20:34:22.878942 18518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:34:22.879091 18518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:34:22.880460 18518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/data/instance:
uuid: "4d447759b8a34705a8151859213d2193"
format_stamp: "Formatted at 2026-06-19 20:34:22 on dist-test-slave-4m58"
server_key: "ee612ade79fb144c4ad06d0207ffe843"
server_key_iv: "5bb68a09f94ea3845152be8ff57283c4"
server_key_version: "encryptionkey@0"
I20260619 20:34:22.880841 18518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance:
uuid: "4d447759b8a34705a8151859213d2193"
format_stamp: "Formatted at 2026-06-19 20:34:22 on dist-test-slave-4m58"
server_key: "ee612ade79fb144c4ad06d0207ffe843"
server_key_iv: "5bb68a09f94ea3845152be8ff57283c4"
server_key_version: "encryptionkey@0"
I20260619 20:34:22.882428 18518 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:34:22.883373 18533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:22.883617 18518 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:34:22.883715 18518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "4d447759b8a34705a8151859213d2193"
format_stamp: "Formatted at 2026-06-19 20:34:22 on dist-test-slave-4m58"
server_key: "ee612ade79fb144c4ad06d0207ffe843"
server_key_iv: "5bb68a09f94ea3845152be8ff57283c4"
server_key_version: "encryptionkey@0"
I20260619 20:34:22.883786 18518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:34:22.898557 18518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:34:22.901170 18518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:34:22.901345 18518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:34:22.901654 18518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:34:22.902004 18518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:34:22.902040 18518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:22.902073 18518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:34:22.902098 18518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:22.909947 18518 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:44993
I20260619 20:34:22.910027 18647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:44993 every 8 connection(s)
I20260619 20:34:22.910444 18518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
I20260619 20:34:22.915814 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 18518
I20260619 20:34:22.915954 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance
I20260619 20:34:22.916143 4429 external_mini_cluster.cc:1428] Setting key c44b00f453d13e6660fa47282dd5c269
Jun 19 20:34:22 dist-test-slave-4m58 krb5kdc[18141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901262, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:34:22.921537 18232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:22.911505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:52777 (local address 127.4.83.126:41283)
0619 20:34:22.911617 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:22.911622 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:22.911920 (+ 298us) server_negotiation.cc:407] Connection header received
0619 20:34:22.912456 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:22.912458 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:22.912472 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:22.912549 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:22.913547 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:22.914105 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:22.915245 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:22.915484 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:22.915548 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:22.918560 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:22.918562 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:22.918563 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:22.918573 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:22.920000 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:22.920505 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:22.920506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:22.920507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:22.920568 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:22.920814 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:22.920815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:22.920815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:22.920959 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:22.921019 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:22.921331 (+ 312us) server_negotiation.cc:299] Negotiation successful
0619 20:34:22.921430 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 20:34:22.921751 18650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:22.911441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.67:52777)
0619 20:34:22.911651 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:22.911697 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:22.912330 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:22.912610 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:22.912614 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:22.913239 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:34:22.913460 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:22.913465 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:22.914199 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:22.914200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:22.915161 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:22.915164 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:22.915589 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:22.915590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:22.915715 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:22.916694 (+ 979us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:34:22.916709 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:34:22.918454 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:34:22.920105 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:22.920116 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:22.920119 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:22.920413 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:22.920628 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:22.920628 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:22.920629 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:22.920755 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:22.921016 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:34:22.921024 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:34:22.921196 (+ 172us) client_negotiation.cc:769] Sending connection context
0619 20:34:22.921297 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 20:34:22.921499 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":93,"threads_started":1}
I20260619 20:34:22.922333 18648 heartbeater.cc:347] Connected to a master server at 127.4.83.126:41283
I20260619 20:34:22.922438 18648 heartbeater.cc:464] Registering TS with master...
I20260619 20:34:22.922650 18648 heartbeater.cc:511] Master 127.4.83.126:41283 requested a full tablet report, sending...
I20260619 20:34:22.923100 18186 ts_manager.cc:194] Registered new tserver with Master: 4d447759b8a34705a8151859213d2193 (127.4.83.67:44993)
I20260619 20:34:22.923497 18186 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.67@KRBTEST.COM'} at 127.4.83.67:52777
I20260619 20:34:22.928586 4429 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:34:22.936810 18232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:22.930083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44052 (local address 127.4.83.126:41283)
0619 20:34:22.930144 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:22.930147 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:22.930209 (+ 62us) server_negotiation.cc:407] Connection header received
0619 20:34:22.930294 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:22.930295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:22.930307 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:22.930357 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:22.931003 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:22.931545 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:22.932438 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:22.932658 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:22.932721 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:22.933958 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:22.933960 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:22.933960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:22.933970 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:22.935463 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:22.935789 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:22.935790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:22.935790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:22.935908 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:22.936130 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:22.936130 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:22.936131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:22.936346 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:22.936436 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:22.936602 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 20:34:22.936725 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 20:34:22.938642 18186 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44052:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:34:22.940718 18186 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:34:22.948231 18659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:22.943719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:44361 (local address 127.0.0.1:56812)
0619 20:34:22.944457 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:22.944466 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:22.944526 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:22.944713 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:22.944714 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:22.944717 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:34:22.944781 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:22.944783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:22.945721 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:22.945722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:22.946833 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:22.946836 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:22.947886 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:22.947887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:22.947971 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:22.947988 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 20:34:22.948061 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 20:34:22.948138 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"mutex_wait_us":5,"thread_start_us":95,"threads_started":1}
I20260619 20:34:22.948243 18664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:22.944022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56812 (local address 127.4.83.65:44361)
0619 20:34:22.944240 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:22.944244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:22.944519 (+ 275us) server_negotiation.cc:407] Connection header received
0619 20:34:22.944600 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:22.944603 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:22.944625 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:22.944706 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:34:22.944844 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:22.945617 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:22.946910 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:22.947663 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:22.947738 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:22.947822 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:22.948080 (+ 258us) server_negotiation.cc:299] Negotiation successful
0619 20:34:22.948130 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":102,"threads_started":1}
I20260619 20:34:22.948336 18662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:22.943689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54068 (local address 127.4.83.66:37693)
0619 20:34:22.943942 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:22.943946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:22.944158 (+ 212us) server_negotiation.cc:407] Connection header received
0619 20:34:22.944215 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:22.944218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:22.944236 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:22.944352 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:34:22.944604 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:22.945832 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:22.947110 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:22.947862 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:22.947929 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:22.947985 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:22.948138 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 20:34:22.948223 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":110,"threads_started":1}
I20260619 20:34:22.948678 18660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:22.943493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:37693 (local address 127.0.0.1:54068)
0619 20:34:22.943939 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:22.943972 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:22.944060 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:22.944366 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:22.944370 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:22.944373 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:34:22.944471 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:22.944475 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:22.945948 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:22.945949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:22.946999 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:22.947003 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:22.947946 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:22.947947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:22.948014 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:22.948019 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:34:22.948523 (+ 504us) client_negotiation.cc:240] Negotiation successful
0619 20:34:22.948599 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":2,"thread_start_us":110,"threads_started":1}
I20260619 20:34:22.949344 18312 tablet_service.cc:1467] Processing CreateTablet for tablet b6c95119b3204edd9f75a982b7ee71b8 (DEFAULT_TABLE table=test-table [id=6983c876509a4d1e823ae98acc236f84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:34:22.949471 18663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:22.943491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:44993 (local address 127.0.0.1:35734)
0619 20:34:22.943910 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:22.943956 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:22.944099 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:22.944502 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:22.944503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:22.944506 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:34:22.944548 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:22.944549 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:22.945948 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:22.945949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:22.947721 (+ 1772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:22.947723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:22.949231 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:22.949232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:22.949300 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:22.949306 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 20:34:22.949356 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 20:34:22.949399 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":976,"thread_start_us":80,"threads_started":1}
I20260619 20:34:22.949492 18661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:22.943593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35734 (local address 127.4.83.67:44993)
0619 20:34:22.943773 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:22.943778 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:22.944014 (+ 236us) server_negotiation.cc:407] Connection header received
0619 20:34:22.944301 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:22.944304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:22.944327 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:22.944398 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:34:22.944625 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:22.945814 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:22.947814 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:22.949148 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:22.949212 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:22.949278 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:22.949369 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 20:34:22.949411 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":90,"threads_started":1}
I20260619 20:34:22.949787 18312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6c95119b3204edd9f75a982b7ee71b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:34:22.950325 18582 tablet_service.cc:1467] Processing CreateTablet for tablet b6c95119b3204edd9f75a982b7ee71b8 (DEFAULT_TABLE table=test-table [id=6983c876509a4d1e823ae98acc236f84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:34:22.950654 18582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6c95119b3204edd9f75a982b7ee71b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:34:22.953156 18665 tablet_bootstrap.cc:492] T b6c95119b3204edd9f75a982b7ee71b8 P 307138c79bd84bb5b8d3791eecfe5940: Bootstrap starting.
I20260619 20:34:22.954044 18665 tablet_bootstrap.cc:654] T b6c95119b3204edd9f75a982b7ee71b8 P 307138c79bd84bb5b8d3791eecfe5940: Neither blocks nor log segments found. Creating new log.
I20260619 20:34:22.954136 18666 tablet_bootstrap.cc:492] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193: Bootstrap starting.
I20260619 20:34:22.954169 18447 tablet_service.cc:1467] Processing CreateTablet for tablet b6c95119b3204edd9f75a982b7ee71b8 (DEFAULT_TABLE table=test-table [id=6983c876509a4d1e823ae98acc236f84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:34:22.954535 18447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6c95119b3204edd9f75a982b7ee71b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:34:22.954816 18665 log.cc:824] T b6c95119b3204edd9f75a982b7ee71b8 P 307138c79bd84bb5b8d3791eecfe5940: Log is configured to *not* fsync() on all Append() calls
I20260619 20:34:22.955030 18666 tablet_bootstrap.cc:654] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193: Neither blocks nor log segments found. Creating new log.
I20260619 20:34:22.955593 18666 log.cc:824] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193: Log is configured to *not* fsync() on all Append() calls
I20260619 20:34:22.955931 18665 tablet_bootstrap.cc:492] T b6c95119b3204edd9f75a982b7ee71b8 P 307138c79bd84bb5b8d3791eecfe5940: No bootstrap required, opened a new log
I20260619 20:34:22.956033 18665 ts_tablet_manager.cc:1397] T b6c95119b3204edd9f75a982b7ee71b8 P 307138c79bd84bb5b8d3791eecfe5940: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 20:34:22.956359 18666 tablet_bootstrap.cc:492] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193: No bootstrap required, opened a new log
I20260619 20:34:22.956434 18666 ts_tablet_manager.cc:1397] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:34:22.957993 18669 tablet_bootstrap.cc:492] T b6c95119b3204edd9f75a982b7ee71b8 P 0e4f581737d04471b15ec0dbcdd5a5fa: Bootstrap starting.
I20260619 20:34:22.958177 18666 raft_consensus.cc:348] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37693 } } peers { permanent_uuid: "307138c79bd84bb5b8d3791eecfe5940" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 44361 } } peers { permanent_uuid: "4d447759b8a34705a8151859213d2193" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44993 } }
I20260619 20:34:22.958401 18666 raft_consensus.cc:374] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:34:22.958446 18666 raft_consensus.cc:729] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d447759b8a34705a8151859213d2193, State: Initialized, Role: FOLLOWER
I20260619 20:34:22.958226 18665 raft_consensus.cc:348] T b6c95119b3204edd9f75a982b7ee71b8 P 307138c79bd84bb5b8d3791eecfe5940 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37693 } } peers { permanent_uuid: "307138c79bd84bb5b8d3791eecfe5940" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 44361 } } peers { permanent_uuid: "4d447759b8a34705a8151859213d2193" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44993 } }
I20260619 20:34:22.958493 18665 raft_consensus.cc:374] T b6c95119b3204edd9f75a982b7ee71b8 P 307138c79bd84bb5b8d3791eecfe5940 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:34:22.958532 18665 raft_consensus.cc:729] T b6c95119b3204edd9f75a982b7ee71b8 P 307138c79bd84bb5b8d3791eecfe5940 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 307138c79bd84bb5b8d3791eecfe5940, State: Initialized, Role: FOLLOWER
I20260619 20:34:22.958585 18666 consensus_queue.cc:260] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37693 } } peers { permanent_uuid: "307138c79bd84bb5b8d3791eecfe5940" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 44361 } } peers { permanent_uuid: "4d447759b8a34705a8151859213d2193" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44993 } }
I20260619 20:34:22.958683 18665 consensus_queue.cc:260] T b6c95119b3204edd9f75a982b7ee71b8 P 307138c79bd84bb5b8d3791eecfe5940 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37693 } } peers { permanent_uuid: "307138c79bd84bb5b8d3791eecfe5940" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 44361 } } peers { permanent_uuid: "4d447759b8a34705a8151859213d2193" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44993 } }
I20260619 20:34:22.958949 18666 ts_tablet_manager.cc:1428] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 20:34:22.958992 18665 ts_tablet_manager.cc:1428] T b6c95119b3204edd9f75a982b7ee71b8 P 307138c79bd84bb5b8d3791eecfe5940: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:34:22.959101 18648 heartbeater.cc:503] Master 127.4.83.126:41283 was elected leader, sending a full tablet report...
I20260619 20:34:22.959120 18378 heartbeater.cc:503] Master 127.4.83.126:41283 was elected leader, sending a full tablet report...
I20260619 20:34:22.959889 18669 tablet_bootstrap.cc:654] T b6c95119b3204edd9f75a982b7ee71b8 P 0e4f581737d04471b15ec0dbcdd5a5fa: Neither blocks nor log segments found. Creating new log.
I20260619 20:34:22.960723 18669 log.cc:824] T b6c95119b3204edd9f75a982b7ee71b8 P 0e4f581737d04471b15ec0dbcdd5a5fa: Log is configured to *not* fsync() on all Append() calls
I20260619 20:34:22.961768 18669 tablet_bootstrap.cc:492] T b6c95119b3204edd9f75a982b7ee71b8 P 0e4f581737d04471b15ec0dbcdd5a5fa: No bootstrap required, opened a new log
I20260619 20:34:22.961869 18669 ts_tablet_manager.cc:1397] T b6c95119b3204edd9f75a982b7ee71b8 P 0e4f581737d04471b15ec0dbcdd5a5fa: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 20:34:22.963637 18669 raft_consensus.cc:348] T b6c95119b3204edd9f75a982b7ee71b8 P 0e4f581737d04471b15ec0dbcdd5a5fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37693 } } peers { permanent_uuid: "307138c79bd84bb5b8d3791eecfe5940" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 44361 } } peers { permanent_uuid: "4d447759b8a34705a8151859213d2193" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44993 } }
I20260619 20:34:22.963943 18669 raft_consensus.cc:374] T b6c95119b3204edd9f75a982b7ee71b8 P 0e4f581737d04471b15ec0dbcdd5a5fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:34:22.963976 18669 raft_consensus.cc:729] T b6c95119b3204edd9f75a982b7ee71b8 P 0e4f581737d04471b15ec0dbcdd5a5fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e4f581737d04471b15ec0dbcdd5a5fa, State: Initialized, Role: FOLLOWER
I20260619 20:34:22.964169 18669 consensus_queue.cc:260] T b6c95119b3204edd9f75a982b7ee71b8 P 0e4f581737d04471b15ec0dbcdd5a5fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37693 } } peers { permanent_uuid: "307138c79bd84bb5b8d3791eecfe5940" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 44361 } } peers { permanent_uuid: "4d447759b8a34705a8151859213d2193" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44993 } }
I20260619 20:34:22.964552 18669 ts_tablet_manager.cc:1428] T b6c95119b3204edd9f75a982b7ee71b8 P 0e4f581737d04471b15ec0dbcdd5a5fa: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:34:22.964644 18513 heartbeater.cc:503] Master 127.4.83.126:41283 was elected leader, sending a full tablet report...
W20260619 20:34:22.992744 18514 tablet.cc:2369] T b6c95119b3204edd9f75a982b7ee71b8 P 0e4f581737d04471b15ec0dbcdd5a5fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:34:23.058267 18379 tablet.cc:2369] T b6c95119b3204edd9f75a982b7ee71b8 P 307138c79bd84bb5b8d3791eecfe5940: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:34:23.114898 18670 raft_consensus.cc:482] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:34:23.115082 18670 raft_consensus.cc:504] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37693 } } peers { permanent_uuid: "307138c79bd84bb5b8d3791eecfe5940" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 44361 } } peers { permanent_uuid: "4d447759b8a34705a8151859213d2193" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44993 } }
I20260619 20:34:23.115607 18670 leader_election.cc:288] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e4f581737d04471b15ec0dbcdd5a5fa (127.4.83.66:37693), 307138c79bd84bb5b8d3791eecfe5940 (127.4.83.65:44361)
I20260619 20:34:23.118830 18662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:23.116019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:34831 (local address 127.4.83.66:37693)
0619 20:34:23.116083 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:23.116084 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:23.116096 (+ 12us) server_negotiation.cc:407] Connection header received
0619 20:34:23.116156 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:23.116157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:23.116170 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:23.116220 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:34:23.116439 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.116961 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.118017 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.118449 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.118516 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:23.118532 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:23.118684 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 20:34:23.118732 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 20:34:23.118830 18650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:23.115880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:37693 (local address 127.4.83.67:34831)
0619 20:34:23.115968 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:23.115991 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:23.116086 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:23.116251 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:23.116252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:23.116255 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:34:23.116343 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:23.116345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.117062 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.117063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:23.117919 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:23.117921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.118532 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.118534 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:23.118619 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:23.118623 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:34:23.118679 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 20:34:23.118738 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":9}
I20260619 20:34:23.118917 18664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:23.116116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:38839 (local address 127.4.83.65:44361)
0619 20:34:23.116176 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:23.116179 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:23.116356 (+ 177us) server_negotiation.cc:407] Connection header received
0619 20:34:23.116393 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:23.116394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:23.116407 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:23.116458 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:34:23.116642 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.117142 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.118199 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.118569 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.118629 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:23.118642 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:23.118790 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 20:34:23.118830 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 20:34:23.118988 18674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:23.116019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:44361 (local address 127.4.83.67:38839)
0619 20:34:23.116276 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:23.116286 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:23.116340 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:23.116487 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:23.116488 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:23.116490 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:34:23.116579 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:23.116582 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.117224 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.117225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:23.118102 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:23.118105 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.118643 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.118643 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:23.118726 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:23.118731 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:34:23.118776 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 20:34:23.118830 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":170,"threads_started":1}
I20260619 20:34:23.119349 18332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6c95119b3204edd9f75a982b7ee71b8" candidate_uuid: "4d447759b8a34705a8151859213d2193" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "307138c79bd84bb5b8d3791eecfe5940" is_pre_election: true
I20260619 20:34:23.119433 18467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6c95119b3204edd9f75a982b7ee71b8" candidate_uuid: "4d447759b8a34705a8151859213d2193" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" is_pre_election: true
I20260619 20:34:23.119586 18467 raft_consensus.cc:2454] T b6c95119b3204edd9f75a982b7ee71b8 P 0e4f581737d04471b15ec0dbcdd5a5fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d447759b8a34705a8151859213d2193 in term 0.
I20260619 20:34:23.119586 18332 raft_consensus.cc:2454] T b6c95119b3204edd9f75a982b7ee71b8 P 307138c79bd84bb5b8d3791eecfe5940 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d447759b8a34705a8151859213d2193 in term 0.
I20260619 20:34:23.119885 18537 leader_election.cc:302] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 307138c79bd84bb5b8d3791eecfe5940, 4d447759b8a34705a8151859213d2193; no voters:
I20260619 20:34:23.120034 18670 raft_consensus.cc:2781] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:34:23.120093 18670 raft_consensus.cc:482] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:34:23.120127 18670 raft_consensus.cc:3037] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:34:23.120971 18670 raft_consensus.cc:504] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37693 } } peers { permanent_uuid: "307138c79bd84bb5b8d3791eecfe5940" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 44361 } } peers { permanent_uuid: "4d447759b8a34705a8151859213d2193" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44993 } }
I20260619 20:34:23.121109 18670 leader_election.cc:288] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193 [CANDIDATE]: Term 1 election: Requested vote from peers 0e4f581737d04471b15ec0dbcdd5a5fa (127.4.83.66:37693), 307138c79bd84bb5b8d3791eecfe5940 (127.4.83.65:44361)
I20260619 20:34:23.121387 18332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6c95119b3204edd9f75a982b7ee71b8" candidate_uuid: "4d447759b8a34705a8151859213d2193" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "307138c79bd84bb5b8d3791eecfe5940"
I20260619 20:34:23.121368 18467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6c95119b3204edd9f75a982b7ee71b8" candidate_uuid: "4d447759b8a34705a8151859213d2193" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa"
I20260619 20:34:23.121474 18332 raft_consensus.cc:3037] T b6c95119b3204edd9f75a982b7ee71b8 P 307138c79bd84bb5b8d3791eecfe5940 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:34:23.121517 18467 raft_consensus.cc:3037] T b6c95119b3204edd9f75a982b7ee71b8 P 0e4f581737d04471b15ec0dbcdd5a5fa [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:34:23.122323 18332 raft_consensus.cc:2454] T b6c95119b3204edd9f75a982b7ee71b8 P 307138c79bd84bb5b8d3791eecfe5940 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d447759b8a34705a8151859213d2193 in term 1.
I20260619 20:34:23.122407 18467 raft_consensus.cc:2454] T b6c95119b3204edd9f75a982b7ee71b8 P 0e4f581737d04471b15ec0dbcdd5a5fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d447759b8a34705a8151859213d2193 in term 1.
I20260619 20:34:23.122509 18537 leader_election.cc:302] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 307138c79bd84bb5b8d3791eecfe5940, 4d447759b8a34705a8151859213d2193; no voters:
I20260619 20:34:23.122639 18670 raft_consensus.cc:2781] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:34:23.122834 18670 raft_consensus.cc:686] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193 [term 1 LEADER]: Becoming Leader. State: Replica: 4d447759b8a34705a8151859213d2193, State: Running, Role: LEADER
I20260619 20:34:23.122947 18670 consensus_queue.cc:237] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37693 } } peers { permanent_uuid: "307138c79bd84bb5b8d3791eecfe5940" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 44361 } } peers { permanent_uuid: "4d447759b8a34705a8151859213d2193" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44993 } }
I20260619 20:34:23.124125 18185 catalog_manager.cc:5495] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d447759b8a34705a8151859213d2193 (127.4.83.67). New cstate: current_term: 1 leader_uuid: "4d447759b8a34705a8151859213d2193" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "307138c79bd84bb5b8d3791eecfe5940" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 44361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d447759b8a34705a8151859213d2193" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44993 } health_report { overall_health: HEALTHY } } }
I20260619 20:34:23.135035 18661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:23.132347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35738 (local address 127.4.83.67:44993)
0619 20:34:23.132447 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:23.132450 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:23.132461 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:34:23.132503 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:23.132504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:23.132528 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:23.132610 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:34:23.132806 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.133318 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.134159 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.134368 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.134442 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:23.134636 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:34:23.134749 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:34:23.134813 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:23.134899 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 20:34:23.134921 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 20:34:23.137527 18467 raft_consensus.cc:1261] T b6c95119b3204edd9f75a982b7ee71b8 P 0e4f581737d04471b15ec0dbcdd5a5fa [term 1 FOLLOWER]: Refusing update from remote peer 4d447759b8a34705a8151859213d2193: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:34:23.137527 18332 raft_consensus.cc:1261] T b6c95119b3204edd9f75a982b7ee71b8 P 307138c79bd84bb5b8d3791eecfe5940 [term 1 FOLLOWER]: Refusing update from remote peer 4d447759b8a34705a8151859213d2193: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:34:23.138025 18675 consensus_queue.cc:1035] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193 [LEADER]: Connected to new peer: Peer: permanent_uuid: "307138c79bd84bb5b8d3791eecfe5940" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:34:23.138221 18670 consensus_queue.cc:1035] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:34:23.142333 18680 mvcc.cc:205] Tried to move back new op lower bound from 7298667573807517696 to 7298667573752393728. Current Snapshot: MvccSnapshot[applied={T|T < 7298667573807517696 or (T in {7298667573807517696})}]
I20260619 20:34:23.143800 18185 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:34:23.143924 18185 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:34:23.144732 18185 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3: Sending DeleteTablet for 3 replicas of tablet b6c95119b3204edd9f75a982b7ee71b8
I20260619 20:34:23.145083 18582 tablet_service.cc:1514] Processing DeleteTablet for tablet b6c95119b3204edd9f75a982b7ee71b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:34:23 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:35734
I20260619 20:34:23.145188 18312 tablet_service.cc:1514] Processing DeleteTablet for tablet b6c95119b3204edd9f75a982b7ee71b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:34:23 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:56812
I20260619 20:34:23.145325 18689 tablet_replica.cc:330] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193: stopping tablet replica
I20260619 20:34:23.145303 18447 tablet_service.cc:1514] Processing DeleteTablet for tablet b6c95119b3204edd9f75a982b7ee71b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:34:23 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:54068
I20260619 20:34:23.145417 18689 raft_consensus.cc:2229] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:34:23.145524 18689 raft_consensus.cc:2258] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:34:23.145598 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 18248
I20260619 20:34:23.146883 18689 ts_tablet_manager.cc:1905] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:34:23.148764 18689 ts_tablet_manager.cc:1918] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:34:23.148849 18689 log.cc:1196] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/wal/wals/b6c95119b3204edd9f75a982b7ee71b8
I20260619 20:34:23.149122 18689 ts_tablet_manager.cc:1939] T b6c95119b3204edd9f75a982b7ee71b8 P 4d447759b8a34705a8151859213d2193: Deleting consensus metadata
I20260619 20:34:23.149945 18172 catalog_manager.cc:4841] TS 4d447759b8a34705a8151859213d2193 (127.4.83.67:44993): tablet b6c95119b3204edd9f75a982b7ee71b8 (table test-table [id=6983c876509a4d1e823ae98acc236f84]) successfully deleted
W20260619 20:34:23.152010 18173 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.83.65:44361 (error 108)
I20260619 20:34:23.152177 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 18383
I20260619 20:34:23.153203 18663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:23.153090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:44361 (local address 127.0.0.1:56814)
0619 20:34:23.153138 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:23.153166 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.65:44361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260619 20:34:23.153316 18173 catalog_manager.cc:4584] TS 307138c79bd84bb5b8d3791eecfe5940 (127.4.83.65:44361): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b6c95119b3204edd9f75a982b7ee71b8: Network error: Client connection negotiation failed: client connection to 127.4.83.65:44361: connect: Connection refused (error 111)
I20260619 20:34:23.159173 18663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:23.159049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:37693 (local address 127.0.0.1:54082)
0619 20:34:23.159099 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:23.159129 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.66:37693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260619 20:34:23.159442 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 18518
I20260619 20:34:23.165143 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 18154
I20260619 20:34:23.171389 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:41283
--webserver_interface=127.4.83.126
--webserver_port=35161
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:33145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:41283
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:34:23.269289 18693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:34:23.269583 18693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:34:23.269610 18693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:34:23.271106 18693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:34:23.271174 18693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:34:23.271189 18693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:34:23.271201 18693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:34:23.271214 18693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:34:23.273187 18693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:33145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:41283
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:41283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=35161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:34:23.273545 18693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:34:23.273852 18693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:34:23.284189 18699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:23.284333 18700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:23.284356 18693 server_base.cc:1034] running on GCE node
W20260619 20:34:23.284374 18703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:23.284854 18693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:34:23.285246 18693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:34:23.286464 18693 hybrid_clock.cc:648] HybridClock initialized: now 1781901263286446 us; error 42 us; skew 500 ppm
Jun 19 20:34:23 dist-test-slave-4m58 krb5kdc[18141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901263, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:34:23.292846 18693 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:34:23.293440 18693 webserver.cc:460] Webserver started at http://127.4.83.126:35161/ using document root <none> and password file <none>
I20260619 20:34:23.293694 18693 fs_manager.cc:359] Metadata directory not provided
I20260619 20:34:23.293764 18693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:34:23.295522 18693 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 20:34:23.296516 18710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:23.296866 18693 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:34:23.297016 18693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "f6176e2299aa4187ad26ef24af4f4ba3"
format_stamp: "Formatted at 2026-06-19 20:34:22 on dist-test-slave-4m58"
server_key: "9b80028812366155afb87c71b4374df3"
server_key_iv: "51387129ce274ef1c1cb26144ac8aa9c"
server_key_version: "encryptionkey@0"
I20260619 20:34:23.297488 18693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:34:23.329594 18693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:34:23.331888 18693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:34:23.332044 18693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:34:23.336750 18693 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:41283
I20260619 20:34:23.336939 18762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:41283 every 8 connection(s)
I20260619 20:34:23.337229 18693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:34:23.338238 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 18693
I20260619 20:34:23.338668 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:44361
--local_ip_for_outbound_sockets=127.4.83.65
--tserver_master_addrs=127.4.83.126:41283
--webserver_port=40069
--webserver_interface=127.4.83.65
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:33145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:34:23.339599 18763 sys_catalog.cc:263] Verifying existing consensus state
I20260619 20:34:23.340574 18763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3: Bootstrap starting.
I20260619 20:34:23.350937 18763 log.cc:824] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3: Log is configured to *not* fsync() on all Append() calls
I20260619 20:34:23.352993 18763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:34:23.353288 18763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3: Bootstrap complete.
I20260619 20:34:23.355420 18763 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6176e2299aa4187ad26ef24af4f4ba3" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 41283 } }
I20260619 20:34:23.355660 18763 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6176e2299aa4187ad26ef24af4f4ba3, State: Initialized, Role: FOLLOWER
I20260619 20:34:23.355818 18763 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6176e2299aa4187ad26ef24af4f4ba3" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 41283 } }
I20260619 20:34:23.355934 18763 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:34:23.356001 18763 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:34:23.356053 18763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [term 1 FOLLOWER]: Advancing to term 2
I20260619 20:34:23.357106 18763 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6176e2299aa4187ad26ef24af4f4ba3" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 41283 } }
I20260619 20:34:23.357205 18763 leader_election.cc:302] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6176e2299aa4187ad26ef24af4f4ba3; no voters:
I20260619 20:34:23.357381 18763 leader_election.cc:288] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 20:34:23.357434 18767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 20:34:23.357546 18767 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [term 2 LEADER]: Becoming Leader. State: Replica: f6176e2299aa4187ad26ef24af4f4ba3, State: Running, Role: LEADER
I20260619 20:34:23.357677 18763 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:34:23.357658 18767 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6176e2299aa4187ad26ef24af4f4ba3" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 41283 } }
I20260619 20:34:23.357985 18768 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f6176e2299aa4187ad26ef24af4f4ba3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6176e2299aa4187ad26ef24af4f4ba3" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 41283 } } }
I20260619 20:34:23.358079 18768 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [sys.catalog]: This master's current role is: LEADER
I20260619 20:34:23.358031 18769 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6176e2299aa4187ad26ef24af4f4ba3. Latest consensus state: current_term: 2 leader_uuid: "f6176e2299aa4187ad26ef24af4f4ba3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6176e2299aa4187ad26ef24af4f4ba3" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 41283 } } }
I20260619 20:34:23.358551 18769 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [sys.catalog]: This master's current role is: LEADER
I20260619 20:34:23.358785 18776 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:34:23.359884 18776 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:34:23.360001 18776 catalog_manager.cc:1259] Loaded cluster ID: ecdff7059dd640ddaa527d1be8940d7d
I20260619 20:34:23.360035 18776 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:34:23.360313 18776 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260619 20:34:23.365180 18776 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260619 20:34:23.368274 18776 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:34:23.368547 18776 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3: Loaded TSK: 0
I20260619 20:34:23.368768 18776 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 20:34:23.444605 18765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:34:23.444918 18765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:34:23.444953 18765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:34:23.446630 18765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:34:23.446725 18765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:34:23.446761 18765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:34:23.449065 18765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:33145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:44361
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=40069
--tserver_master_addrs=127.4.83.126:41283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:34:23.449437 18765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:34:23.449719 18765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:34:23.455802 18794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:23.455899 18791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:23.455904 18792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:23.456395 18765 server_base.cc:1034] running on GCE node
I20260619 20:34:23.456683 18765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:34:23.457011 18765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:34:23.458184 18765 hybrid_clock.cc:648] HybridClock initialized: now 1781901263458151 us; error 58 us; skew 500 ppm
Jun 19 20:34:23 dist-test-slave-4m58 krb5kdc[18141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901263, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:34:23.461802 18765 init.cc:377] Logged in from keytab as kudu/127.4.83.65@KRBTEST.COM (short username kudu)
I20260619 20:34:23.462466 18765 webserver.cc:460] Webserver started at http://127.4.83.65:40069/ using document root <none> and password file <none>
I20260619 20:34:23.462723 18765 fs_manager.cc:359] Metadata directory not provided
I20260619 20:34:23.462800 18765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:34:23.464664 18765 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:34:23.465554 18801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:23.465881 18765 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:34:23.466043 18765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "307138c79bd84bb5b8d3791eecfe5940"
format_stamp: "Formatted at 2026-06-19 20:34:22 on dist-test-slave-4m58"
server_key: "d775174f80afc71a3ce59d737b51cfd8"
server_key_iv: "e69b93b97fb23c33350661ac3b91cee2"
server_key_version: "encryptionkey@0"
I20260619 20:34:23.466598 18765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:34:23.492244 18765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:34:23.495119 18765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:34:23.495276 18765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:34:23.495563 18765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:34:23.496263 18808 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 20:34:23.498208 18765 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 20:34:23.498308 18765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 20:34:23.498347 18765 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 20:34:23.499349 18765 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 20:34:23.499394 18765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 20:34:23.499579 18808 tablet_bootstrap.cc:492] T b6c95119b3204edd9f75a982b7ee71b8 P 307138c79bd84bb5b8d3791eecfe5940: Bootstrap starting.
I20260619 20:34:23.507145 18765 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:44361
I20260619 20:34:23.507329 18916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:44361 every 8 connection(s)
I20260619 20:34:23.507629 18765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
I20260619 20:34:23.508911 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 18765
I20260619 20:34:23.509203 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:37693
--local_ip_for_outbound_sockets=127.4.83.66
--tserver_master_addrs=127.4.83.126:41283
--webserver_port=37767
--webserver_interface=127.4.83.66
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:33145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:34:23.513156 18808 log.cc:824] T b6c95119b3204edd9f75a982b7ee71b8 P 307138c79bd84bb5b8d3791eecfe5940: Log is configured to *not* fsync() on all Append() calls
I20260619 20:34:23.516717 18808 tablet_bootstrap.cc:492] T b6c95119b3204edd9f75a982b7ee71b8 P 307138c79bd84bb5b8d3791eecfe5940: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:34:23.517058 18808 tablet_bootstrap.cc:492] T b6c95119b3204edd9f75a982b7ee71b8 P 307138c79bd84bb5b8d3791eecfe5940: Bootstrap complete.
I20260619 20:34:23.517983 18808 ts_tablet_manager.cc:1397] T b6c95119b3204edd9f75a982b7ee71b8 P 307138c79bd84bb5b8d3791eecfe5940: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.015s
I20260619 20:34:23.519686 18808 raft_consensus.cc:348] T b6c95119b3204edd9f75a982b7ee71b8 P 307138c79bd84bb5b8d3791eecfe5940 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37693 } } peers { permanent_uuid: "307138c79bd84bb5b8d3791eecfe5940" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 44361 } } peers { permanent_uuid: "4d447759b8a34705a8151859213d2193" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44993 } }
I20260619 20:34:23.520021 18808 raft_consensus.cc:729] T b6c95119b3204edd9f75a982b7ee71b8 P 307138c79bd84bb5b8d3791eecfe5940 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 307138c79bd84bb5b8d3791eecfe5940, State: Initialized, Role: FOLLOWER
I20260619 20:34:23.520217 18808 consensus_queue.cc:260] T b6c95119b3204edd9f75a982b7ee71b8 P 307138c79bd84bb5b8d3791eecfe5940 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37693 } } peers { permanent_uuid: "307138c79bd84bb5b8d3791eecfe5940" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 44361 } } peers { permanent_uuid: "4d447759b8a34705a8151859213d2193" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44993 } }
I20260619 20:34:23.520578 18808 ts_tablet_manager.cc:1428] T b6c95119b3204edd9f75a982b7ee71b8 P 307138c79bd84bb5b8d3791eecfe5940: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
Jun 19 20:34:23 dist-test-slave-4m58 krb5kdc[18141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901263, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:34:23.527767 18919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:23.508664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.65:40521)
0619 20:34:23.513648 (+ 4984us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:23.513702 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:23.514515 (+ 813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:23.515266 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:23.515271 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:23.515705 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:34:23.516103 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:23.516113 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.517878 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.517880 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:23.519256 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:23.519260 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.520382 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.520385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:23.520570 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:23.521203 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:34:23.521214 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:34:23.523018 (+ 1804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:34:23.525562 (+ 2544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:23.525577 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:23.525580 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:23.525957 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:23.526298 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:23.526299 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:23.526299 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:23.526444 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:23.526937 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:34:23.526949 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:34:23.527140 (+ 191us) client_negotiation.cc:769] Sending connection context
0619 20:34:23.527299 (+ 159us) client_negotiation.cc:240] Negotiation successful
0619 20:34:23.527521 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4928,"thread_start_us":142,"threads_started":1}
I20260619 20:34:23.527773 18920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:23.508754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:40521 (local address 127.4.83.126:41283)
0619 20:34:23.510586 (+ 1832us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:23.510601 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:23.513916 (+ 3315us) server_negotiation.cc:407] Connection header received
0619 20:34:23.514876 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:23.514897 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:23.514984 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:23.515203 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:23.516282 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.517711 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.519388 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.520108 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.520245 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:23.523190 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:23.523196 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:23.523197 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:23.523211 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:23.525419 (+ 2208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:23.526074 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:23.526078 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:23.526078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:23.526213 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:23.526510 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:23.526510 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:23.526511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:23.526825 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:23.526929 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:23.527375 (+ 446us) server_negotiation.cc:299] Negotiation successful
0619 20:34:23.527548 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1754,"thread_start_us":102,"threads_started":1}
I20260619 20:34:23.528892 18917 heartbeater.cc:347] Connected to a master server at 127.4.83.126:41283
I20260619 20:34:23.529047 18917 heartbeater.cc:464] Registering TS with master...
I20260619 20:34:23.529289 18917 heartbeater.cc:511] Master 127.4.83.126:41283 requested a full tablet report, sending...
I20260619 20:34:23.530315 18727 ts_manager.cc:194] Registered new tserver with Master: 307138c79bd84bb5b8d3791eecfe5940 (127.4.83.65:44361)
I20260619 20:34:23.531591 18727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.65@KRBTEST.COM'} at 127.4.83.65:40521
I20260619 20:34:23.532758 18917 heartbeater.cc:503] Master 127.4.83.126:41283 was elected leader, sending a full tablet report...
Jun 19 20:34:23 dist-test-slave-4m58 krb5kdc[18141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901263, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for kudu/127.4.83.65@KRBTEST.COM
I20260619 20:34:23.540773 18924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:23.531082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:44361 (local address 127.0.0.1:56828)
0619 20:34:23.531436 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:23.531481 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:23.531590 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:23.531867 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:23.531870 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:23.532247 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:34:23.532316 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:23.532319 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.533277 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.533278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:23.533976 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:23.533979 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.534489 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.534490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:23.534580 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:23.535147 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:34:23.535153 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:34:23.537087 (+ 1934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:34:23.539170 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:23.539187 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:23.539190 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:23.539488 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:23.539734 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:23.539735 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:23.539735 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:23.539808 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:23.540181 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:34:23.540191 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:34:23.540267 (+ 76us) client_negotiation.cc:769] Sending connection context
0619 20:34:23.540342 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 20:34:23.540537 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":113,"threads_started":1}
I20260619 20:34:23.540773 18925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:23.531190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56828 (local address 127.4.83.65:44361)
0619 20:34:23.531430 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:23.531435 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:23.531580 (+ 145us) server_negotiation.cc:407] Connection header received
0619 20:34:23.531701 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:23.531704 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:23.531720 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:23.531774 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:23.532404 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.533197 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.534066 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.534409 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.534478 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:23.537227 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:23.537242 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:23.537242 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:23.537252 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:23.539037 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:23.539589 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:23.539593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:23.539593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:23.539670 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:23.539902 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:23.539903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:23.539903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:23.540112 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:23.540197 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:23.540408 (+ 211us) server_negotiation.cc:299] Negotiation successful
0619 20:34:23.540532 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":122,"threads_started":1}
I20260619 20:34:23.541528 18850 tablet_service.cc:1514] Processing DeleteTablet for tablet b6c95119b3204edd9f75a982b7ee71b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:34:23 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:56828
I20260619 20:34:23.542210 18926 tablet_replica.cc:330] stopping tablet replica
I20260619 20:34:23.542357 18926 raft_consensus.cc:2229] T b6c95119b3204edd9f75a982b7ee71b8 P 307138c79bd84bb5b8d3791eecfe5940 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:34:23.542443 18926 raft_consensus.cc:2258] T b6c95119b3204edd9f75a982b7ee71b8 P 307138c79bd84bb5b8d3791eecfe5940 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:34:23.542833 18926 ts_tablet_manager.cc:1905] T b6c95119b3204edd9f75a982b7ee71b8 P 307138c79bd84bb5b8d3791eecfe5940: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:34:23.545409 18926 ts_tablet_manager.cc:1918] T b6c95119b3204edd9f75a982b7ee71b8 P 307138c79bd84bb5b8d3791eecfe5940: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:34:23.545527 18926 log.cc:1196] T b6c95119b3204edd9f75a982b7ee71b8 P 307138c79bd84bb5b8d3791eecfe5940: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/wal/wals/b6c95119b3204edd9f75a982b7ee71b8
I20260619 20:34:23.545756 18926 ts_tablet_manager.cc:1939] T b6c95119b3204edd9f75a982b7ee71b8 P 307138c79bd84bb5b8d3791eecfe5940: Deleting consensus metadata
I20260619 20:34:23.546435 18714 catalog_manager.cc:4841] TS 307138c79bd84bb5b8d3791eecfe5940 (127.4.83.65:44361): tablet b6c95119b3204edd9f75a982b7ee71b8 (table test-table [id=6983c876509a4d1e823ae98acc236f84]) successfully deleted
W20260619 20:34:23.599903 18921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:34:23.600185 18921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:34:23.600224 18921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:34:23.601472 18921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:34:23.601547 18921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:34:23.601583 18921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:34:23.603093 18921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:33145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:37693
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=37767
--tserver_master_addrs=127.4.83.126:41283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:34:23.603327 18921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:34:23.603530 18921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:34:23.609659 18934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:23.609670 18932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:23.609910 18931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:23.610123 18921 server_base.cc:1034] running on GCE node
I20260619 20:34:23.610343 18921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:34:23.610627 18921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:34:23.611800 18921 hybrid_clock.cc:648] HybridClock initialized: now 1781901263611739 us; error 79 us; skew 500 ppm
Jun 19 20:34:23 dist-test-slave-4m58 krb5kdc[18141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901263, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:34:23.614488 18921 init.cc:377] Logged in from keytab as kudu/127.4.83.66@KRBTEST.COM (short username kudu)
I20260619 20:34:23.614979 18921 webserver.cc:460] Webserver started at http://127.4.83.66:37767/ using document root <none> and password file <none>
I20260619 20:34:23.615183 18921 fs_manager.cc:359] Metadata directory not provided
I20260619 20:34:23.615242 18921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:34:23.616851 18921 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:34:23.617825 18941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:23.618103 18921 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:34:23.618208 18921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "0e4f581737d04471b15ec0dbcdd5a5fa"
format_stamp: "Formatted at 2026-06-19 20:34:22 on dist-test-slave-4m58"
server_key: "c0c3b6184e7bf83659f4f26efaee3c20"
server_key_iv: "f9c03385c9f44da91e936158f8bdce69"
server_key_version: "encryptionkey@0"
I20260619 20:34:23.618585 18921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:34:23.635881 18921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:34:23.638062 18921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:34:23.638180 18921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:34:23.638386 18921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:34:23.638885 18948 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 20:34:23.640781 18921 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 20:34:23.640872 18921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 20:34:23.640944 18921 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 20:34:23.641755 18921 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 20:34:23.641793 18921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 20:34:23.641949 18948 tablet_bootstrap.cc:492] T b6c95119b3204edd9f75a982b7ee71b8 P 0e4f581737d04471b15ec0dbcdd5a5fa: Bootstrap starting.
I20260619 20:34:23.649178 18921 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:37693
I20260619 20:34:23.649442 19056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:37693 every 8 connection(s)
I20260619 20:34:23.649585 18921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
I20260619 20:34:23.654803 18948 log.cc:824] T b6c95119b3204edd9f75a982b7ee71b8 P 0e4f581737d04471b15ec0dbcdd5a5fa: Log is configured to *not* fsync() on all Append() calls
I20260619 20:34:23.655349 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 18921
I20260619 20:34:23.655638 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:44993
--local_ip_for_outbound_sockets=127.4.83.67
--tserver_master_addrs=127.4.83.126:41283
--webserver_port=39573
--webserver_interface=127.4.83.67
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:33145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 20:34:23 dist-test-slave-4m58 krb5kdc[18141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901263, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:34:23.658265 18948 tablet_bootstrap.cc:492] T b6c95119b3204edd9f75a982b7ee71b8 P 0e4f581737d04471b15ec0dbcdd5a5fa: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:34:23.658860 18948 tablet_bootstrap.cc:492] T b6c95119b3204edd9f75a982b7ee71b8 P 0e4f581737d04471b15ec0dbcdd5a5fa: Bootstrap complete.
I20260619 20:34:23.661394 18920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:23.650623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:47309 (local address 127.4.83.126:41283)
0619 20:34:23.650718 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:23.650720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:23.650842 (+ 122us) server_negotiation.cc:407] Connection header received
0619 20:34:23.651369 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:23.651370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:23.651391 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:23.651533 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:23.652263 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.653142 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.654277 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.654630 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.654747 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:23.657654 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:23.657656 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:23.657657 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:23.657670 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:23.659696 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:23.660228 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:23.660229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:23.660229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:23.660325 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:23.660559 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:23.660559 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:23.660560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:23.660724 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:23.660799 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:23.661137 (+ 338us) server_negotiation.cc:299] Negotiation successful
0619 20:34:23.661278 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 20:34:23.661410 19059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:23.650497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.66:47309)
0619 20:34:23.650684 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:23.650729 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:23.651239 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:23.651610 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:23.651614 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:23.651916 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:34:23.652141 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:23.652147 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.653307 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.653308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:23.654155 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:23.654157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.654785 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.654786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:23.654911 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:23.655645 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:34:23.655656 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:34:23.657518 (+ 1862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:34:23.659825 (+ 2307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:23.659855 (+ 30us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:23.659857 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:23.660140 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:23.660393 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:23.660394 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:23.660394 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:23.660489 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:23.660787 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:34:23.660790 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:34:23.660928 (+ 138us) client_negotiation.cc:769] Sending connection context
0619 20:34:23.661071 (+ 143us) client_negotiation.cc:240] Negotiation successful
0619 20:34:23.661229 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":98,"threads_started":1}
I20260619 20:34:23.664713 19057 heartbeater.cc:347] Connected to a master server at 127.4.83.126:41283
I20260619 20:34:23.664844 19057 heartbeater.cc:464] Registering TS with master...
I20260619 20:34:23.664981 18948 ts_tablet_manager.cc:1397] T b6c95119b3204edd9f75a982b7ee71b8 P 0e4f581737d04471b15ec0dbcdd5a5fa: Time spent bootstrapping tablet: real 0.023s user 0.009s sys 0.008s
I20260619 20:34:23.665066 19057 heartbeater.cc:511] Master 127.4.83.126:41283 requested a full tablet report, sending...
I20260619 20:34:23.665910 18727 ts_manager.cc:194] Registered new tserver with Master: 0e4f581737d04471b15ec0dbcdd5a5fa (127.4.83.66:37693)
I20260619 20:34:23.666175 18948 raft_consensus.cc:348] T b6c95119b3204edd9f75a982b7ee71b8 P 0e4f581737d04471b15ec0dbcdd5a5fa [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37693 } } peers { permanent_uuid: "307138c79bd84bb5b8d3791eecfe5940" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 44361 } } peers { permanent_uuid: "4d447759b8a34705a8151859213d2193" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44993 } }
I20260619 20:34:23.666491 18948 raft_consensus.cc:729] T b6c95119b3204edd9f75a982b7ee71b8 P 0e4f581737d04471b15ec0dbcdd5a5fa [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e4f581737d04471b15ec0dbcdd5a5fa, State: Initialized, Role: FOLLOWER
I20260619 20:34:23.666649 18948 consensus_queue.cc:260] T b6c95119b3204edd9f75a982b7ee71b8 P 0e4f581737d04471b15ec0dbcdd5a5fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37693 } } peers { permanent_uuid: "307138c79bd84bb5b8d3791eecfe5940" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 44361 } } peers { permanent_uuid: "4d447759b8a34705a8151859213d2193" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44993 } }
I20260619 20:34:23.666682 18727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.66@KRBTEST.COM'} at 127.4.83.66:47309
I20260619 20:34:23.666891 18948 ts_tablet_manager.cc:1428] T b6c95119b3204edd9f75a982b7ee71b8 P 0e4f581737d04471b15ec0dbcdd5a5fa: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260619 20:34:23.668092 19057 heartbeater.cc:503] Master 127.4.83.126:41283 was elected leader, sending a full tablet report...
Jun 19 20:34:23 dist-test-slave-4m58 krb5kdc[18141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901263, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for kudu/127.4.83.66@KRBTEST.COM
I20260619 20:34:23.674641 18924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:23.666376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:37693 (local address 127.0.0.1:54084)
0619 20:34:23.666852 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:23.666871 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:23.666963 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:23.667299 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:23.667301 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:23.667487 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:34:23.667546 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:23.667549 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.668615 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.668616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:23.669156 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:23.669158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.669651 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.669652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:23.669715 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:23.670122 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:34:23.670127 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:34:23.671670 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:34:23.673650 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:23.673651 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:23.673652 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:23.673819 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:23.674014 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:23.674015 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:23.674015 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:23.674072 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:23.674322 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:34:23.674323 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:34:23.674369 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 20:34:23.674431 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 20:34:23.674536 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449}
I20260619 20:34:23.674646 19062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:23.666499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54084 (local address 127.4.83.66:37693)
0619 20:34:23.667140 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:23.667144 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:23.667155 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:34:23.667184 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:23.667185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:23.667197 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:23.667274 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:23.667653 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.668546 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.669229 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.669584 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.669644 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:23.671741 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:23.671749 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:23.671749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:23.671756 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:23.673557 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:23.673893 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:23.673896 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:23.673897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:23.673954 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:23.674119 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:23.674119 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:23.674120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:23.674264 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:23.674332 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:23.674466 (+ 134us) server_negotiation.cc:299] Negotiation successful
0619 20:34:23.674562 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":96,"threads_started":1}
I20260619 20:34:23.675173 18991 tablet_service.cc:1514] Processing DeleteTablet for tablet b6c95119b3204edd9f75a982b7ee71b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:34:23 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:54084
I20260619 20:34:23.675990 19064 tablet_replica.cc:330] stopping tablet replica
I20260619 20:34:23.676112 19064 raft_consensus.cc:2229] T b6c95119b3204edd9f75a982b7ee71b8 P 0e4f581737d04471b15ec0dbcdd5a5fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:34:23.676187 19064 raft_consensus.cc:2258] T b6c95119b3204edd9f75a982b7ee71b8 P 0e4f581737d04471b15ec0dbcdd5a5fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:34:23.676483 19064 ts_tablet_manager.cc:1905] T b6c95119b3204edd9f75a982b7ee71b8 P 0e4f581737d04471b15ec0dbcdd5a5fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:34:23.678399 19064 ts_tablet_manager.cc:1918] T b6c95119b3204edd9f75a982b7ee71b8 P 0e4f581737d04471b15ec0dbcdd5a5fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:34:23.678465 19064 log.cc:1196] T b6c95119b3204edd9f75a982b7ee71b8 P 0e4f581737d04471b15ec0dbcdd5a5fa: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/wal/wals/b6c95119b3204edd9f75a982b7ee71b8
I20260619 20:34:23.678683 19064 ts_tablet_manager.cc:1939] T b6c95119b3204edd9f75a982b7ee71b8 P 0e4f581737d04471b15ec0dbcdd5a5fa: Deleting consensus metadata
I20260619 20:34:23.679184 18714 catalog_manager.cc:4841] TS 0e4f581737d04471b15ec0dbcdd5a5fa (127.4.83.66:37693): tablet b6c95119b3204edd9f75a982b7ee71b8 (table test-table [id=6983c876509a4d1e823ae98acc236f84]) successfully deleted
W20260619 20:34:23.755579 19061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:34:23.755901 19061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:34:23.755944 19061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:34:23.757333 19061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:34:23.757376 19061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:34:23.757403 19061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:34:23.759042 19061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:33145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:44993
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=39573
--tserver_master_addrs=127.4.83.126:41283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:34:23.759357 19061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:34:23.759614 19061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:34:23.765221 19070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:23.765221 19069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:23.765261 19072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:23.765249 19061 server_base.cc:1034] running on GCE node
I20260619 20:34:23.765723 19061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:34:23.766003 19061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:34:23.767174 19061 hybrid_clock.cc:648] HybridClock initialized: now 1781901263767163 us; error 30 us; skew 500 ppm
Jun 19 20:34:23 dist-test-slave-4m58 krb5kdc[18141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901263, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:34:23.770412 19061 init.cc:377] Logged in from keytab as kudu/127.4.83.67@KRBTEST.COM (short username kudu)
I20260619 20:34:23.771107 19061 webserver.cc:460] Webserver started at http://127.4.83.67:39573/ using document root <none> and password file <none>
I20260619 20:34:23.771354 19061 fs_manager.cc:359] Metadata directory not provided
I20260619 20:34:23.771428 19061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:34:23.773123 19061 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:34:23.773833 19079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:23.774091 19061 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:34:23.774227 19061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "4d447759b8a34705a8151859213d2193"
format_stamp: "Formatted at 2026-06-19 20:34:22 on dist-test-slave-4m58"
server_key: "ee612ade79fb144c4ad06d0207ffe843"
server_key_iv: "5bb68a09f94ea3845152be8ff57283c4"
server_key_version: "encryptionkey@0"
I20260619 20:34:23.774669 19061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:34:23.787305 19061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:34:23.790388 19061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:34:23.790607 19061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:34:23.790897 19061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:34:23.791311 19061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:34:23.791357 19061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:23.791392 19061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:34:23.791416 19061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:23.799219 19061 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:44993
I20260619 20:34:23.799345 19193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:44993 every 8 connection(s)
I20260619 20:34:23.799741 19061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
I20260619 20:34:23.803499 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 19061
Jun 19 20:34:23 dist-test-slave-4m58 krb5kdc[18141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901263, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:34:23.812418 19197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:23.804066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44062 (local address 127.4.83.126:41283)
0619 20:34:23.804237 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:23.804240 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:23.804373 (+ 133us) server_negotiation.cc:407] Connection header received
0619 20:34:23.804434 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:23.804435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:23.804451 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:23.804590 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:23.805539 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.806399 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.807318 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.807591 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.807657 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:23.809374 (+ 1717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:23.809376 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:23.809376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:23.809384 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:23.810872 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:23.811338 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:23.811339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:23.811339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:23.811412 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:23.811619 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:23.811620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:23.811620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:23.811937 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:23.812058 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:23.812167 (+ 109us) server_negotiation.cc:299] Negotiation successful
0619 20:34:23.812295 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":86,"threads_started":1}
I20260619 20:34:23.812700 18920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:23.800658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:41623 (local address 127.4.83.126:41283)
0619 20:34:23.800739 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:23.800742 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:23.801055 (+ 313us) server_negotiation.cc:407] Connection header received
0619 20:34:23.801735 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:23.801736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:23.801751 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:23.801827 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:23.803022 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.803565 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.805164 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.805520 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.805594 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:23.808704 (+ 3110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:23.808706 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:23.808706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:23.808848 (+ 142us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:23.810721 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:23.811384 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:23.811385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:23.811386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:23.811454 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:23.811799 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:23.811800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:23.811800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:23.812012 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:23.812094 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:23.812497 (+ 403us) server_negotiation.cc:299] Negotiation successful
0619 20:34:23.812603 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 20:34:23.812995 19196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:23.800664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.67:41623)
0619 20:34:23.800888 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:23.800939 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:23.801617 (+ 678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:23.801921 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:23.801927 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:23.802624 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:34:23.802891 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:23.802897 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.803741 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.803743 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:23.804995 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:23.805000 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.805664 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.805665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:23.805902 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:23.806693 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:34:23.806703 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:34:23.808572 (+ 1869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:34:23.810868 (+ 2296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:23.810888 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:23.810893 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:23.811278 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:23.811538 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:23.811539 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:23.811539 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:23.811720 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:23.812096 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:34:23.812103 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:34:23.812324 (+ 221us) client_negotiation.cc:769] Sending connection context
0619 20:34:23.812468 (+ 144us) client_negotiation.cc:240] Negotiation successful
0619 20:34:23.812691 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":118,"threads_started":1}
I20260619 20:34:23.813546 19194 heartbeater.cc:347] Connected to a master server at 127.4.83.126:41283
I20260619 20:34:23.813657 19194 heartbeater.cc:464] Registering TS with master...
I20260619 20:34:23.813853 19194 heartbeater.cc:511] Master 127.4.83.126:41283 requested a full tablet report, sending...
I20260619 20:34:23.814296 18727 ts_manager.cc:194] Registered new tserver with Master: 4d447759b8a34705a8151859213d2193 (127.4.83.67:44993)
I20260619 20:34:23.814694 18727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.67@KRBTEST.COM'} at 127.4.83.67:41623
I20260619 20:34:23.824990 4429 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:34:23.828197 18920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:23.825462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44070 (local address 127.4.83.126:41283)
0619 20:34:23.825555 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:23.825557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:23.825572 (+ 15us) server_negotiation.cc:407] Connection header received
0619 20:34:23.825670 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:23.825671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:23.825724 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:23.825809 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:34:23.825942 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.826492 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.827308 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.827512 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.827573 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:23.827718 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:34:23.827907 (+ 189us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:34:23.827996 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:23.828064 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 20:34:23.828089 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 20:34:23.828667 18727 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:44070:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:34:23.830798 18727 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:34:23.833689 18850 tablet_service.cc:1467] Processing CreateTablet for tablet ba36d0425f6540dcbfda972ab93aa88f (DEFAULT_TABLE table=test-table [id=ea90c47ab978479594237730677c982b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:34:23.833702 18991 tablet_service.cc:1467] Processing CreateTablet for tablet ba36d0425f6540dcbfda972ab93aa88f (DEFAULT_TABLE table=test-table [id=ea90c47ab978479594237730677c982b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:34:23.833922 18991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba36d0425f6540dcbfda972ab93aa88f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:34:23.833923 18850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba36d0425f6540dcbfda972ab93aa88f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:34:23.835575 18808 tablet_bootstrap.cc:492] T ba36d0425f6540dcbfda972ab93aa88f P 307138c79bd84bb5b8d3791eecfe5940: Bootstrap starting.
I20260619 20:34:23.835659 18948 tablet_bootstrap.cc:492] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa: Bootstrap starting.
I20260619 20:34:23.835979 18808 tablet_bootstrap.cc:654] T ba36d0425f6540dcbfda972ab93aa88f P 307138c79bd84bb5b8d3791eecfe5940: Neither blocks nor log segments found. Creating new log.
I20260619 20:34:23.836138 18948 tablet_bootstrap.cc:654] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa: Neither blocks nor log segments found. Creating new log.
I20260619 20:34:23.836532 18808 tablet_bootstrap.cc:492] T ba36d0425f6540dcbfda972ab93aa88f P 307138c79bd84bb5b8d3791eecfe5940: No bootstrap required, opened a new log
I20260619 20:34:23.836573 18808 ts_tablet_manager.cc:1397] T ba36d0425f6540dcbfda972ab93aa88f P 307138c79bd84bb5b8d3791eecfe5940: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 20:34:23.837086 18948 tablet_bootstrap.cc:492] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa: No bootstrap required, opened a new log
I20260619 20:34:23.836946 18808 raft_consensus.cc:348] T ba36d0425f6540dcbfda972ab93aa88f P 307138c79bd84bb5b8d3791eecfe5940 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d447759b8a34705a8151859213d2193" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44993 } } peers { permanent_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37693 } } peers { permanent_uuid: "307138c79bd84bb5b8d3791eecfe5940" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 44361 } }
I20260619 20:34:23.837148 18948 ts_tablet_manager.cc:1397] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:34:23.837136 18808 raft_consensus.cc:374] T ba36d0425f6540dcbfda972ab93aa88f P 307138c79bd84bb5b8d3791eecfe5940 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:34:23.837222 18808 raft_consensus.cc:729] T ba36d0425f6540dcbfda972ab93aa88f P 307138c79bd84bb5b8d3791eecfe5940 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 307138c79bd84bb5b8d3791eecfe5940, State: Initialized, Role: FOLLOWER
I20260619 20:34:23.837284 18808 consensus_queue.cc:260] T ba36d0425f6540dcbfda972ab93aa88f P 307138c79bd84bb5b8d3791eecfe5940 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d447759b8a34705a8151859213d2193" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44993 } } peers { permanent_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37693 } } peers { permanent_uuid: "307138c79bd84bb5b8d3791eecfe5940" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 44361 } }
I20260619 20:34:23.837381 18808 ts_tablet_manager.cc:1428] T ba36d0425f6540dcbfda972ab93aa88f P 307138c79bd84bb5b8d3791eecfe5940: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260619 20:34:23.837338 18948 raft_consensus.cc:348] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d447759b8a34705a8151859213d2193" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44993 } } peers { permanent_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37693 } } peers { permanent_uuid: "307138c79bd84bb5b8d3791eecfe5940" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 44361 } }
I20260619 20:34:23.837426 18948 raft_consensus.cc:374] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:34:23.837493 18948 raft_consensus.cc:729] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e4f581737d04471b15ec0dbcdd5a5fa, State: Initialized, Role: FOLLOWER
I20260619 20:34:23.837551 18948 consensus_queue.cc:260] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d447759b8a34705a8151859213d2193" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44993 } } peers { permanent_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37693 } } peers { permanent_uuid: "307138c79bd84bb5b8d3791eecfe5940" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 44361 } }
I20260619 20:34:23.837642 18948 ts_tablet_manager.cc:1428] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 20:34:23.838039 18924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:23.833395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:44993 (local address 127.0.0.1:35742)
0619 20:34:23.833449 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:23.833462 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:23.833533 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:23.834267 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:23.834269 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:23.834271 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:34:23.834342 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:23.834344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.835333 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.835334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:23.836489 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:23.836492 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.837746 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.837747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:23.837838 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:23.837859 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 20:34:23.837893 (+ 34us) client_negotiation.cc:240] Negotiation successful
0619 20:34:23.837960 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 20:34:23.838392 19200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:23.833609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35742 (local address 127.4.83.67:44993)
0619 20:34:23.834070 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:23.834076 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:23.834089 (+ 13us) server_negotiation.cc:407] Connection header received
0619 20:34:23.834134 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:23.834136 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:23.834154 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:23.834248 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:34:23.834451 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.835229 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.836622 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:23.837621 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:23.838176 (+ 555us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:23.838248 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:23.838275 (+ 27us) server_negotiation.cc:299] Negotiation successful
0619 20:34:23.838316 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":120,"threads_started":1}
I20260619 20:34:23.839147 19127 tablet_service.cc:1467] Processing CreateTablet for tablet ba36d0425f6540dcbfda972ab93aa88f (DEFAULT_TABLE table=test-table [id=ea90c47ab978479594237730677c982b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:34:23.839483 19127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba36d0425f6540dcbfda972ab93aa88f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:34:23.842499 19203 tablet_bootstrap.cc:492] T ba36d0425f6540dcbfda972ab93aa88f P 4d447759b8a34705a8151859213d2193: Bootstrap starting.
I20260619 20:34:23.843482 19203 tablet_bootstrap.cc:654] T ba36d0425f6540dcbfda972ab93aa88f P 4d447759b8a34705a8151859213d2193: Neither blocks nor log segments found. Creating new log.
I20260619 20:34:23.844048 19203 log.cc:824] T ba36d0425f6540dcbfda972ab93aa88f P 4d447759b8a34705a8151859213d2193: Log is configured to *not* fsync() on all Append() calls
I20260619 20:34:23.845034 19203 tablet_bootstrap.cc:492] T ba36d0425f6540dcbfda972ab93aa88f P 4d447759b8a34705a8151859213d2193: No bootstrap required, opened a new log
I20260619 20:34:23.845148 19203 ts_tablet_manager.cc:1397] T ba36d0425f6540dcbfda972ab93aa88f P 4d447759b8a34705a8151859213d2193: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 20:34:23.846980 19203 raft_consensus.cc:348] T ba36d0425f6540dcbfda972ab93aa88f P 4d447759b8a34705a8151859213d2193 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d447759b8a34705a8151859213d2193" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44993 } } peers { permanent_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37693 } } peers { permanent_uuid: "307138c79bd84bb5b8d3791eecfe5940" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 44361 } }
I20260619 20:34:23.847169 19203 raft_consensus.cc:374] T ba36d0425f6540dcbfda972ab93aa88f P 4d447759b8a34705a8151859213d2193 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:34:23.847205 19203 raft_consensus.cc:729] T ba36d0425f6540dcbfda972ab93aa88f P 4d447759b8a34705a8151859213d2193 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d447759b8a34705a8151859213d2193, State: Initialized, Role: FOLLOWER
I20260619 20:34:23.847347 19203 consensus_queue.cc:260] T ba36d0425f6540dcbfda972ab93aa88f P 4d447759b8a34705a8151859213d2193 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d447759b8a34705a8151859213d2193" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44993 } } peers { permanent_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37693 } } peers { permanent_uuid: "307138c79bd84bb5b8d3791eecfe5940" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 44361 } }
I20260619 20:34:23.847658 19203 ts_tablet_manager.cc:1428] T ba36d0425f6540dcbfda972ab93aa88f P 4d447759b8a34705a8151859213d2193: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 20:34:23.847784 19194 heartbeater.cc:503] Master 127.4.83.126:41283 was elected leader, sending a full tablet report...
W20260619 20:34:23.900291 19058 tablet.cc:2369] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:34:24.014031 18918 tablet.cc:2369] T ba36d0425f6540dcbfda972ab93aa88f P 307138c79bd84bb5b8d3791eecfe5940: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:34:24.050644 19195 tablet.cc:2369] T ba36d0425f6540dcbfda972ab93aa88f P 4d447759b8a34705a8151859213d2193: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:34:24.089826 19063 raft_consensus.cc:482] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:34:24.090019 19063 raft_consensus.cc:504] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d447759b8a34705a8151859213d2193" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44993 } } peers { permanent_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37693 } } peers { permanent_uuid: "307138c79bd84bb5b8d3791eecfe5940" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 44361 } }
I20260619 20:34:24.090392 19063 leader_election.cc:288] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d447759b8a34705a8151859213d2193 (127.4.83.67:44993), 307138c79bd84bb5b8d3791eecfe5940 (127.4.83.65:44361)
I20260619 20:34:24.094142 19059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:24.090700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:44361 (local address 127.4.83.66:54221)
0619 20:34:24.090774 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:24.090790 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:24.090880 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:24.091182 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:24.091183 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:24.091185 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:34:24.091267 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:24.091270 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:24.092163 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:24.092164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:24.093018 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:24.093021 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:24.093808 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:24.093809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:24.093939 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:24.093970 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 20:34:24.094031 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 20:34:24.094077 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260619 20:34:24.094245 19206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:24.090838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:54221 (local address 127.4.83.65:44361)
0619 20:34:24.091039 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:24.091041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:24.091049 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:34:24.091075 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:24.091075 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:24.091098 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:24.091166 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:34:24.091375 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:24.092075 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:24.093139 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:24.093713 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:24.093776 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:24.093850 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:24.094072 (+ 222us) server_negotiation.cc:299] Negotiation successful
0619 20:34:24.094143 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":107,"threads_started":1}
I20260619 20:34:24.094352 19207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:24.090700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:44993 (local address 127.4.83.66:47037)
0619 20:34:24.091176 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:24.091187 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:24.091261 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:24.091405 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:24.091405 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:24.091408 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:34:24.091551 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:24.091553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:24.092382 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:24.092383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:24.093302 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:24.093305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:24.094021 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:24.094022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:24.094107 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:24.094113 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 20:34:24.094194 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 20:34:24.094243 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"mutex_wait_us":269,"thread_start_us":104,"threads_started":1}
I20260619 20:34:24.094347 19200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:24.090833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:47037 (local address 127.4.83.67:44993)
0619 20:34:24.090930 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:24.090933 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:24.091262 (+ 329us) server_negotiation.cc:407] Connection header received
0619 20:34:24.091320 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:24.091321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:24.091334 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:24.091384 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:34:24.091670 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:24.092263 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:24.093449 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:24.093868 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:24.093955 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:24.093967 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:24.094219 (+ 252us) server_negotiation.cc:299] Negotiation successful
0619 20:34:24.094263 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 20:34:24.094625 18871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba36d0425f6540dcbfda972ab93aa88f" candidate_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "307138c79bd84bb5b8d3791eecfe5940" is_pre_election: true
I20260619 20:34:24.094815 18871 raft_consensus.cc:2454] T ba36d0425f6540dcbfda972ab93aa88f P 307138c79bd84bb5b8d3791eecfe5940 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e4f581737d04471b15ec0dbcdd5a5fa in term 0.
I20260619 20:34:24.095073 18945 leader_election.cc:302] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e4f581737d04471b15ec0dbcdd5a5fa, 307138c79bd84bb5b8d3791eecfe5940; no voters:
I20260619 20:34:24.095067 19148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba36d0425f6540dcbfda972ab93aa88f" candidate_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d447759b8a34705a8151859213d2193" is_pre_election: true
I20260619 20:34:24.095218 19063 raft_consensus.cc:2781] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:34:24.095237 19148 raft_consensus.cc:2454] T ba36d0425f6540dcbfda972ab93aa88f P 4d447759b8a34705a8151859213d2193 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e4f581737d04471b15ec0dbcdd5a5fa in term 0.
I20260619 20:34:24.095268 19063 raft_consensus.cc:482] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:34:24.095295 19063 raft_consensus.cc:3037] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:34:24.096093 19063 raft_consensus.cc:504] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d447759b8a34705a8151859213d2193" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44993 } } peers { permanent_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37693 } } peers { permanent_uuid: "307138c79bd84bb5b8d3791eecfe5940" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 44361 } }
I20260619 20:34:24.096244 19063 leader_election.cc:288] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa [CANDIDATE]: Term 1 election: Requested vote from peers 4d447759b8a34705a8151859213d2193 (127.4.83.67:44993), 307138c79bd84bb5b8d3791eecfe5940 (127.4.83.65:44361)
I20260619 20:34:24.096472 18871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba36d0425f6540dcbfda972ab93aa88f" candidate_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "307138c79bd84bb5b8d3791eecfe5940"
I20260619 20:34:24.096501 19148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba36d0425f6540dcbfda972ab93aa88f" candidate_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d447759b8a34705a8151859213d2193"
I20260619 20:34:24.096604 19148 raft_consensus.cc:3037] T ba36d0425f6540dcbfda972ab93aa88f P 4d447759b8a34705a8151859213d2193 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:34:24.096604 18871 raft_consensus.cc:3037] T ba36d0425f6540dcbfda972ab93aa88f P 307138c79bd84bb5b8d3791eecfe5940 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:34:24.097800 18871 raft_consensus.cc:2454] T ba36d0425f6540dcbfda972ab93aa88f P 307138c79bd84bb5b8d3791eecfe5940 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e4f581737d04471b15ec0dbcdd5a5fa in term 1.
I20260619 20:34:24.097800 19148 raft_consensus.cc:2454] T ba36d0425f6540dcbfda972ab93aa88f P 4d447759b8a34705a8151859213d2193 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e4f581737d04471b15ec0dbcdd5a5fa in term 1.
I20260619 20:34:24.098032 18945 leader_election.cc:302] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e4f581737d04471b15ec0dbcdd5a5fa, 307138c79bd84bb5b8d3791eecfe5940; no voters:
I20260619 20:34:24.098147 19063 raft_consensus.cc:2781] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:34:24.098296 19063 raft_consensus.cc:686] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa [term 1 LEADER]: Becoming Leader. State: Replica: 0e4f581737d04471b15ec0dbcdd5a5fa, State: Running, Role: LEADER
I20260619 20:34:24.098388 19063 consensus_queue.cc:237] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d447759b8a34705a8151859213d2193" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44993 } } peers { permanent_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37693 } } peers { permanent_uuid: "307138c79bd84bb5b8d3791eecfe5940" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 44361 } }
I20260619 20:34:24.099314 18727 catalog_manager.cc:5495] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e4f581737d04471b15ec0dbcdd5a5fa (127.4.83.66). New cstate: current_term: 1 leader_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d447759b8a34705a8151859213d2193" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e4f581737d04471b15ec0dbcdd5a5fa" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "307138c79bd84bb5b8d3791eecfe5940" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 44361 } health_report { overall_health: UNKNOWN } } }
I20260619 20:34:24.125902 19062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:24.123389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54090 (local address 127.4.83.66:37693)
0619 20:34:24.123458 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:24.123460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:24.123469 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:34:24.123516 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:24.123517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:24.123540 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:24.123615 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:34:24.123734 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:24.124349 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:24.125184 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:24.125409 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:24.125467 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:24.125588 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:34:24.125725 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:34:24.125783 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:24.125804 (+ 21us) server_negotiation.cc:299] Negotiation successful
0619 20:34:24.125824 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 20:34:24.127880 19148 raft_consensus.cc:1261] T ba36d0425f6540dcbfda972ab93aa88f P 4d447759b8a34705a8151859213d2193 [term 1 FOLLOWER]: Refusing update from remote peer 0e4f581737d04471b15ec0dbcdd5a5fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:34:24.127918 18871 raft_consensus.cc:1261] T ba36d0425f6540dcbfda972ab93aa88f P 307138c79bd84bb5b8d3791eecfe5940 [term 1 FOLLOWER]: Refusing update from remote peer 0e4f581737d04471b15ec0dbcdd5a5fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:34:24.128300 19063 consensus_queue.cc:1035] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "307138c79bd84bb5b8d3791eecfe5940" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:34:24.128460 19208 consensus_queue.cc:1035] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d447759b8a34705a8151859213d2193" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:34:24.131074 19211 mvcc.cc:205] Tried to move back new op lower bound from 7298667577864069120 to 7298667577747419136. Current Snapshot: MvccSnapshot[applied={T|T < 7298667577864069120}]
I20260619 20:34:24.132416 19213 mvcc.cc:205] Tried to move back new op lower bound from 7298667577864069120 to 7298667577747419136. Current Snapshot: MvccSnapshot[applied={T|T < 7298667577864069120}]
I20260619 20:34:24.134194 18727 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:44070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:34:24.134321 18727 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:44070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:34:24.135177 18727 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3: Sending DeleteTablet for 3 replicas of tablet ba36d0425f6540dcbfda972ab93aa88f
I20260619 20:34:24.135572 18850 tablet_service.cc:1514] Processing DeleteTablet for tablet ba36d0425f6540dcbfda972ab93aa88f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:34:24 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:56828
I20260619 20:34:24.135550 19127 tablet_service.cc:1514] Processing DeleteTablet for tablet ba36d0425f6540dcbfda972ab93aa88f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:34:24 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:35742
I20260619 20:34:24.135581 18991 tablet_service.cc:1514] Processing DeleteTablet for tablet ba36d0425f6540dcbfda972ab93aa88f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:34:24 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:54084
I20260619 20:34:24.135813 19064 tablet_replica.cc:330] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa: stopping tablet replica
I20260619 20:34:24.135895 19222 tablet_replica.cc:330] T ba36d0425f6540dcbfda972ab93aa88f P 4d447759b8a34705a8151859213d2193: stopping tablet replica
I20260619 20:34:24.135924 19064 raft_consensus.cc:2229] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:34:24.136031 19064 raft_consensus.cc:2258] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:34:24.136021 19222 raft_consensus.cc:2229] T ba36d0425f6540dcbfda972ab93aa88f P 4d447759b8a34705a8151859213d2193 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:34:24.136157 19222 raft_consensus.cc:2258] T ba36d0425f6540dcbfda972ab93aa88f P 4d447759b8a34705a8151859213d2193 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:34:24.136349 19221 tablet_replica.cc:330] T ba36d0425f6540dcbfda972ab93aa88f P 307138c79bd84bb5b8d3791eecfe5940: stopping tablet replica
I20260619 20:34:24.136421 19221 raft_consensus.cc:2229] T ba36d0425f6540dcbfda972ab93aa88f P 307138c79bd84bb5b8d3791eecfe5940 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:34:24.136485 19221 raft_consensus.cc:2258] T ba36d0425f6540dcbfda972ab93aa88f P 307138c79bd84bb5b8d3791eecfe5940 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:34:24.136596 19064 ts_tablet_manager.cc:1905] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:34:24.136883 19221 ts_tablet_manager.cc:1905] T ba36d0425f6540dcbfda972ab93aa88f P 307138c79bd84bb5b8d3791eecfe5940: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:34:24.138242 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 18765
I20260619 20:34:24.138257 19064 ts_tablet_manager.cc:1918] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:34:24.138319 19064 log.cc:1196] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/wal/wals/ba36d0425f6540dcbfda972ab93aa88f
I20260619 20:34:24.138511 19064 ts_tablet_manager.cc:1939] T ba36d0425f6540dcbfda972ab93aa88f P 0e4f581737d04471b15ec0dbcdd5a5fa: Deleting consensus metadata
I20260619 20:34:24.138875 18714 catalog_manager.cc:4841] TS 0e4f581737d04471b15ec0dbcdd5a5fa (127.4.83.66:37693): tablet ba36d0425f6540dcbfda972ab93aa88f (table test-table [id=ea90c47ab978479594237730677c982b]) successfully deleted
I20260619 20:34:24.139259 19222 ts_tablet_manager.cc:1905] T ba36d0425f6540dcbfda972ab93aa88f P 4d447759b8a34705a8151859213d2193: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:34:24.140875 19222 ts_tablet_manager.cc:1918] T ba36d0425f6540dcbfda972ab93aa88f P 4d447759b8a34705a8151859213d2193: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:34:24.140964 19222 log.cc:1196] T ba36d0425f6540dcbfda972ab93aa88f P 4d447759b8a34705a8151859213d2193: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/wal/wals/ba36d0425f6540dcbfda972ab93aa88f
I20260619 20:34:24.141245 19222 ts_tablet_manager.cc:1939] T ba36d0425f6540dcbfda972ab93aa88f P 4d447759b8a34705a8151859213d2193: Deleting consensus metadata
I20260619 20:34:24.141862 18713 catalog_manager.cc:4841] TS 4d447759b8a34705a8151859213d2193 (127.4.83.67:44993): tablet ba36d0425f6540dcbfda972ab93aa88f (table test-table [id=ea90c47ab978479594237730677c982b]) successfully deleted
W20260619 20:34:24.144520 18714 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.83.65:44361 (error 108)
I20260619 20:34:24.145136 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 18921
I20260619 20:34:24.146275 18924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:24.146106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:44361 (local address 127.0.0.1:56830)
0619 20:34:24.146177 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:24.146231 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.65:44361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260619 20:34:24.146468 18714 catalog_manager.cc:4584] TS 307138c79bd84bb5b8d3791eecfe5940 (127.4.83.65:44361): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ba36d0425f6540dcbfda972ab93aa88f: Network error: Client connection negotiation failed: client connection to 127.4.83.65:44361: connect: Connection refused (error 111)
I20260619 20:34:24.152462 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 19061
I20260619 20:34:24.158118 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 18693
I20260619 20:34:24.163429 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:41283
--webserver_interface=127.4.83.126
--webserver_port=35161
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:33145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:41283
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:34:24.257251 19224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:34:24.257543 19224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:34:24.257572 19224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:34:24.259022 19224 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:34:24.259091 19224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:34:24.259105 19224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:34:24.259119 19224 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:34:24.259130 19224 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:34:24.261092 19224 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:33145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:41283
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:41283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=35161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:34:24.261468 19224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:34:24.261763 19224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:34:24.267231 19229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:24.267321 19232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:24.267411 19230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:24.267727 19224 server_base.cc:1034] running on GCE node
I20260619 20:34:24.268054 19224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:34:24.268404 19224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:34:24.269585 19224 hybrid_clock.cc:648] HybridClock initialized: now 1781901264269569 us; error 36 us; skew 500 ppm
Jun 19 20:34:24 dist-test-slave-4m58 krb5kdc[18141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901264, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:34:24.272305 19224 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:34:24.272771 19224 webserver.cc:460] Webserver started at http://127.4.83.126:35161/ using document root <none> and password file <none>
I20260619 20:34:24.273025 19224 fs_manager.cc:359] Metadata directory not provided
I20260619 20:34:24.273072 19224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:34:24.274699 19224 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:34:24.275583 19239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:24.275866 19224 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:34:24.275976 19224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "f6176e2299aa4187ad26ef24af4f4ba3"
format_stamp: "Formatted at 2026-06-19 20:34:22 on dist-test-slave-4m58"
server_key: "9b80028812366155afb87c71b4374df3"
server_key_iv: "51387129ce274ef1c1cb26144ac8aa9c"
server_key_version: "encryptionkey@0"
I20260619 20:34:24.276468 19224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:34:24.285012 19224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:34:24.287820 19224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:34:24.288209 19224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:34:24.293516 19224 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:41283
I20260619 20:34:24.293569 19292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:41283 every 8 connection(s)
I20260619 20:34:24.294066 19224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:34:24.295955 19293 sys_catalog.cc:263] Verifying existing consensus state
I20260619 20:34:24.296677 19293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3: Bootstrap starting.
I20260619 20:34:24.299553 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 19224
I20260619 20:34:24.299957 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:44361
--local_ip_for_outbound_sockets=127.4.83.65
--tserver_master_addrs=127.4.83.126:41283
--webserver_port=40069
--webserver_interface=127.4.83.65
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:33145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:34:24.306030 19293 log.cc:824] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3: Log is configured to *not* fsync() on all Append() calls
I20260619 20:34:24.309500 19293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:34:24.309880 19293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3: Bootstrap complete.
I20260619 20:34:24.313100 19293 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6176e2299aa4187ad26ef24af4f4ba3" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 41283 } }
I20260619 20:34:24.313438 19293 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6176e2299aa4187ad26ef24af4f4ba3, State: Initialized, Role: FOLLOWER
I20260619 20:34:24.313619 19293 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6176e2299aa4187ad26ef24af4f4ba3" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 41283 } }
I20260619 20:34:24.313699 19293 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:34:24.313740 19293 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:34:24.313796 19293 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [term 2 FOLLOWER]: Advancing to term 3
I20260619 20:34:24.315224 19293 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6176e2299aa4187ad26ef24af4f4ba3" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 41283 } }
I20260619 20:34:24.315347 19293 leader_election.cc:302] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6176e2299aa4187ad26ef24af4f4ba3; no voters:
I20260619 20:34:24.315537 19293 leader_election.cc:288] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 20:34:24.315670 19297 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 20:34:24.315878 19293 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:34:24.315959 19297 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [term 3 LEADER]: Becoming Leader. State: Replica: f6176e2299aa4187ad26ef24af4f4ba3, State: Running, Role: LEADER
I20260619 20:34:24.316102 19297 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6176e2299aa4187ad26ef24af4f4ba3" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 41283 } }
I20260619 20:34:24.316545 19298 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "f6176e2299aa4187ad26ef24af4f4ba3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6176e2299aa4187ad26ef24af4f4ba3" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 41283 } } }
I20260619 20:34:24.316543 19299 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6176e2299aa4187ad26ef24af4f4ba3. Latest consensus state: current_term: 3 leader_uuid: "f6176e2299aa4187ad26ef24af4f4ba3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6176e2299aa4187ad26ef24af4f4ba3" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 41283 } } }
I20260619 20:34:24.316684 19298 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [sys.catalog]: This master's current role is: LEADER
I20260619 20:34:24.316685 19299 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6176e2299aa4187ad26ef24af4f4ba3 [sys.catalog]: This master's current role is: LEADER
I20260619 20:34:24.316972 19306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:34:24.317947 19306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:34:24.318059 19306 catalog_manager.cc:1259] Loaded cluster ID: ecdff7059dd640ddaa527d1be8940d7d
I20260619 20:34:24.318095 19306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260619 20:34:24.318456 19306 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f122c3659ad google::LogMessage::Fail() at ??:0
@ 0x7f122c36888a google::LogMessage::SendToLog() at ??:0
@ 0x7f122c3654de google::LogMessage::Flush() at ??:0
@ 0x7f122c366a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f123428a0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f12342938b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f122e2330d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f122e228954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f122f08c6db start_thread at ??:0
@ 0x7f122ae6361f clone at ??:0
W20260619 20:34:24.423776 19295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:34:24.424150 19295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:34:24.424191 19295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:34:24.426043 19295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:34:24.426126 19295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:34:24.426172 19295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:34:24.428505 19295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:33145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:44361
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=40069
--tserver_master_addrs=127.4.83.126:41283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:34:24.428857 19295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:34:24.429200 19295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:34:24.435266 19329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:24.435400 19330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:24.435429 19332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:24.435784 19295 server_base.cc:1034] running on GCE node
I20260619 20:34:24.436080 19295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:34:24.436404 19295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:34:24.437562 19295 hybrid_clock.cc:648] HybridClock initialized: now 1781901264437529 us; error 53 us; skew 500 ppm
Jun 19 20:34:24 dist-test-slave-4m58 krb5kdc[18141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901264, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:34:24.440582 19295 init.cc:377] Logged in from keytab as kudu/127.4.83.65@KRBTEST.COM (short username kudu)
I20260619 20:34:24.441156 19295 webserver.cc:460] Webserver started at http://127.4.83.65:40069/ using document root <none> and password file <none>
I20260619 20:34:24.441394 19295 fs_manager.cc:359] Metadata directory not provided
I20260619 20:34:24.441447 19295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:34:24.443044 19295 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:34:24.443912 19339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:24.444322 19295 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:34:24.444419 19295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "307138c79bd84bb5b8d3791eecfe5940"
format_stamp: "Formatted at 2026-06-19 20:34:22 on dist-test-slave-4m58"
server_key: "d775174f80afc71a3ce59d737b51cfd8"
server_key_iv: "e69b93b97fb23c33350661ac3b91cee2"
server_key_version: "encryptionkey@0"
I20260619 20:34:24.444871 19295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:34:24.466491 19295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:34:24.469003 19295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:34:24.469179 19295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:34:24.469447 19295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:34:24.470120 19346 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260619 20:34:24.472257 19346 ts_tablet_manager.cc:1852] T ba36d0425f6540dcbfda972ab93aa88f P 307138c79bd84bb5b8d3791eecfe5940: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260619 20:34:24.472687 19346 ts_tablet_manager.cc:1905] T ba36d0425f6540dcbfda972ab93aa88f P 307138c79bd84bb5b8d3791eecfe5940: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:34:24.475883 19346 ts_tablet_manager.cc:1918] T ba36d0425f6540dcbfda972ab93aa88f P 307138c79bd84bb5b8d3791eecfe5940: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:34:24.476022 19346 log.cc:1196] T ba36d0425f6540dcbfda972ab93aa88f P 307138c79bd84bb5b8d3791eecfe5940: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/wal/wals/ba36d0425f6540dcbfda972ab93aa88f
I20260619 20:34:24.476330 19346 ts_tablet_manager.cc:1939] T ba36d0425f6540dcbfda972ab93aa88f P 307138c79bd84bb5b8d3791eecfe5940: Deleting consensus metadata
I20260619 20:34:24.476637 19295 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260619 20:34:24.476740 19295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.000s sys 0.000s
I20260619 20:34:24.476794 19295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:34:24.476815 19295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:24.485280 19295 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:44361
I20260619 20:34:24.485437 19454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:44361 every 8 connection(s)
I20260619 20:34:24.485802 19295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
I20260619 20:34:24.488165 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 19295
I20260619 20:34:24.488485 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:37693
--local_ip_for_outbound_sockets=127.4.83.66
--tserver_master_addrs=127.4.83.126:41283
--webserver_port=37767
--webserver_interface=127.4.83.66
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:33145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:34:24.535465 19457 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:34:24.486702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.65:38067)
0619 20:34:24.486960 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:24.487023 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:24.487979 (+ 956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:24.535187 (+ 47208us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":100,"threads_started":1}
W20260619 20:34:24.535884 19455 heartbeater.cc:644] Failed to heartbeat to 127.4.83.126:41283 (0 consecutive failures): Network error: Failed to ping master at 127.4.83.126:41283: Client connection negotiation failed: client connection to 127.4.83.126:41283: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 20:34:24.536346 19457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:24.536186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.65:57467)
0619 20:34:24.536254 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:24.536318 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260619 20:34:24.536736 19457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:24.536643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.65:36841)
0619 20:34:24.536691 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:24.536719 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 20:34:24.536893 19455 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 20:34:24.582247 19458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:34:24.582556 19458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:34:24.582585 19458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:34:24.584164 19458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:34:24.584230 19458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:34:24.584266 19458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:34:24.586018 19458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:33145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:37693
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=37767
--tserver_master_addrs=127.4.83.126:41283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:34:24.586328 19458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:34:24.586594 19458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:34:24.591998 19463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:24.592008 19466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:24.592103 19464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:24.592372 19458 server_base.cc:1034] running on GCE node
I20260619 20:34:24.592598 19458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:34:24.592871 19458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:34:24.594014 19458 hybrid_clock.cc:648] HybridClock initialized: now 1781901264593973 us; error 57 us; skew 500 ppm
Jun 19 20:34:24 dist-test-slave-4m58 krb5kdc[18141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901264, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:34:24.596824 19458 init.cc:377] Logged in from keytab as kudu/127.4.83.66@KRBTEST.COM (short username kudu)
I20260619 20:34:24.597338 19458 webserver.cc:460] Webserver started at http://127.4.83.66:37767/ using document root <none> and password file <none>
I20260619 20:34:24.597587 19458 fs_manager.cc:359] Metadata directory not provided
I20260619 20:34:24.597642 19458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:34:24.599224 19458 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:34:24.600093 19473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:24.600343 19458 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:34:24.600446 19458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "0e4f581737d04471b15ec0dbcdd5a5fa"
format_stamp: "Formatted at 2026-06-19 20:34:22 on dist-test-slave-4m58"
server_key: "c0c3b6184e7bf83659f4f26efaee3c20"
server_key_iv: "f9c03385c9f44da91e936158f8bdce69"
server_key_version: "encryptionkey@0"
I20260619 20:34:24.600879 19458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:34:24.619629 19458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:34:24.622225 19458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:34:24.622432 19458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:34:24.622700 19458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:34:24.623051 19458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:34:24.623088 19458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:24.623133 19458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:34:24.623159 19458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:24.631093 19458 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:37693
I20260619 20:34:24.631219 19588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:37693 every 8 connection(s)
I20260619 20:34:24.631644 19458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
I20260619 20:34:24.633008 19591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:24.632441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.66:57245)
0619 20:34:24.632713 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:24.632812 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":106,"threads_started":1}
W20260619 20:34:24.633489 19589 heartbeater.cc:644] Failed to heartbeat to 127.4.83.126:41283 (0 consecutive failures): Network error: Failed to ping master at 127.4.83.126:41283: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
I20260619 20:34:24.633841 19591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:24.633728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.66:47985)
0619 20:34:24.633785 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:24.633819 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 20:34:24.634197 19591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:24.634026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.66:54185)
0619 20:34:24.634140 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:24.634174 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260619 20:34:24.634378 19589 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 20:34:24.635387 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 19458
I20260619 20:34:24.635672 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:44993
--local_ip_for_outbound_sockets=127.4.83.67
--tserver_master_addrs=127.4.83.126:41283
--webserver_port=39573
--webserver_interface=127.4.83.67
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:33145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:34:24.721470 19592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:34:24.721684 19592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:34:24.721707 19592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:34:24.723038 19592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:34:24.723095 19592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:34:24.723124 19592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:34:24.724637 19592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:33145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:44993
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=39573
--tserver_master_addrs=127.4.83.126:41283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:34:24.724876 19592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:34:24.725124 19592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:34:24.731541 19597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:24.731546 19598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:24.731684 19592 server_base.cc:1034] running on GCE node
W20260619 20:34:24.731897 19600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:24.732141 19592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:34:24.732414 19592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:34:24.733577 19592 hybrid_clock.cc:648] HybridClock initialized: now 1781901264733560 us; error 39 us; skew 500 ppm
Jun 19 20:34:24 dist-test-slave-4m58 krb5kdc[18141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901264, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:34:24.736577 19592 init.cc:377] Logged in from keytab as kudu/127.4.83.67@KRBTEST.COM (short username kudu)
I20260619 20:34:24.737088 19592 webserver.cc:460] Webserver started at http://127.4.83.67:39573/ using document root <none> and password file <none>
I20260619 20:34:24.737308 19592 fs_manager.cc:359] Metadata directory not provided
I20260619 20:34:24.737370 19592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:34:24.738806 19592 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:34:24.739537 19607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:24.739792 19592 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:34:24.739935 19592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "4d447759b8a34705a8151859213d2193"
format_stamp: "Formatted at 2026-06-19 20:34:22 on dist-test-slave-4m58"
server_key: "ee612ade79fb144c4ad06d0207ffe843"
server_key_iv: "5bb68a09f94ea3845152be8ff57283c4"
server_key_version: "encryptionkey@0"
I20260619 20:34:24.740299 19592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:34:24.752068 19592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:34:24.754303 19592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:34:24.754482 19592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:34:24.754719 19592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:34:24.755112 19592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:34:24.755154 19592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:24.755184 19592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:34:24.755203 19592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:24.761940 19592 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:44993
I20260619 20:34:24.762027 19721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:44993 every 8 connection(s)
I20260619 20:34:24.762404 19592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
I20260619 20:34:24.763933 19724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:24.763315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.67:46991)
0619 20:34:24.763615 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:24.763719 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":114,"threads_started":1}
W20260619 20:34:24.764370 19722 heartbeater.cc:644] Failed to heartbeat to 127.4.83.126:41283 (0 consecutive failures): Network error: Failed to ping master at 127.4.83.126:41283: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
I20260619 20:34:24.764818 19724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:24.764633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.67:54901)
0619 20:34:24.764726 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:24.764773 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260619 20:34:24.765239 19724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:24.765144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.67:43355)
0619 20:34:24.765193 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:24.765218 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 20:34:24.765399 19722 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 20:34:24.771790 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 19592
I20260619 20:34:25.537789 19726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:25.537445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.65:33313)
0619 20:34:25.537656 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:25.537725 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":97,"threads_started":1}
I20260619 20:34:25.635043 19727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:25.634746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.66:40487)
0619 20:34:25.634903 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:25.634956 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":65,"threads_started":1}
I20260619 20:34:25.766161 19728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:25.765796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.67:51087)
0619 20:34:25.765981 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:25.766054 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":79,"threads_started":1}
I20260619 20:34:26.538709 19729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:26.538405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.65:32863)
0619 20:34:26.538594 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:26.538644 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":87,"threads_started":1}
I20260619 20:34:26.635983 19730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:26.635620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.66:40823)
0619 20:34:26.635815 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:26.635906 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":98,"threads_started":1}
I20260619 20:34:26.767163 19731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:26.766819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.67:54751)
0619 20:34:26.767027 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:26.767074 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":102,"threads_started":1}
I20260619 20:34:27.539528 19732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:27.539208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.65:41677)
0619 20:34:27.539394 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:27.539441 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":90,"threads_started":1}
I20260619 20:34:27.636873 19733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:27.636559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.66:49429)
0619 20:34:27.636758 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:27.636803 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":84,"threads_started":1}
I20260619 20:34:27.768181 19734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:27.767864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.67:54109)
0619 20:34:27.768053 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:27.768114 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":98,"threads_started":1}
I20260619 20:34:28.540385 19735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:28.540132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.65:57329)
0619 20:34:28.540280 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:28.540336 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":90,"threads_started":1}
I20260619 20:34:28.637637 19736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:28.637411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.66:34391)
0619 20:34:28.637538 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:28.637582 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":65,"threads_started":1}
I20260619 20:34:28.768942 19737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:28.768701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.67:35259)
0619 20:34:28.768860 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:28.768894 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":81,"threads_started":1}
I20260619 20:34:29.541229 19738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:29.540913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.65:53637)
0619 20:34:29.541119 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:29.541165 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":117,"threads_started":1}
I20260619 20:34:29.638417 19739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:29.638145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.66:45891)
0619 20:34:29.638305 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:29.638349 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":77,"threads_started":1}
I20260619 20:34:29.769877 19740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:29.769529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.67:47425)
0619 20:34:29.769752 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:29.769811 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":85,"threads_started":1}
I20260619 20:34:30.542176 19741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:30.541896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.65:36223)
0619 20:34:30.542054 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:30.542112 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
I20260619 20:34:30.639505 19742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:30.639125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.66:34729)
0619 20:34:30.639382 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:30.639438 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":97,"threads_started":1}
I20260619 20:34:30.770629 19743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:30.770394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.67:56431)
0619 20:34:30.770526 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:30.770563 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":79,"threads_started":1}
I20260619 20:34:31.543164 19744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:31.542879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.65:56969)
0619 20:34:31.543051 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:31.543102 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":94,"threads_started":1}
I20260619 20:34:31.640403 19745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:31.640048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.66:52791)
0619 20:34:31.640274 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:31.640336 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":100,"threads_started":1}
I20260619 20:34:31.771641 19746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:31.771303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.67:35601)
0619 20:34:31.771520 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:31.771573 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":110,"threads_started":1}
I20260619 20:34:32.544106 19747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:32.543743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.65:47929)
0619 20:34:32.543973 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:32.544040 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":139,"threads_started":1}
I20260619 20:34:32.641215 19748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:32.640944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.66:37639)
0619 20:34:32.641098 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:32.641148 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":80,"threads_started":1}
I20260619 20:34:32.772585 19749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:32.772292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.67:54971)
0619 20:34:32.772459 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:32.772519 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":100,"threads_started":1}
I20260619 20:34:33.545060 19750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:33.544702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.65:33803)
0619 20:34:33.544912 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:33.544980 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":102,"threads_started":1}
I20260619 20:34:33.642052 19751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:33.641785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.66:38143)
0619 20:34:33.641934 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:33.641985 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":85,"threads_started":1}
I20260619 20:34:33.773429 19752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:33.773129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.67:58079)
0619 20:34:33.773306 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:33.773362 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":85,"threads_started":1}
I20260619 20:34:34.545998 19753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:34.545662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.65:52035)
0619 20:34:34.545878 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:34.545930 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":102,"threads_started":1}
I20260619 20:34:34.643066 19754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:34.642711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.66:45887)
0619 20:34:34.642891 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:34.642992 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":85,"threads_started":1}
I20260619 20:34:34.774354 19755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:34.774038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.67:38141)
0619 20:34:34.774238 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:34.774290 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":100,"threads_started":1}
I20260619 20:34:35.547252 19756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:35.546788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.65:59497)
0619 20:34:35.547130 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:35.547185 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":220,"threads_started":1}
I20260619 20:34:35.644062 19757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:35.643757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.66:40605)
0619 20:34:35.643939 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:35.643994 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":108,"threads_started":1}
I20260619 20:34:35.775377 19758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:35.775026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.67:35527)
0619 20:34:35.775229 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:35.775307 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":88,"threads_started":1}
I20260619 20:34:36.548393 19759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:36.548009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.65:35739)
0619 20:34:36.548274 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:36.548329 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":98,"threads_started":1}
I20260619 20:34:36.644996 19761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:36.644690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.66:47445)
0619 20:34:36.644873 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:36.644929 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":103,"threads_started":1}
I20260619 20:34:36.776487 19762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:36.776160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.67:33667)
0619 20:34:36.776354 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:36.776419 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":114,"threads_started":1}
I20260619 20:34:37.549244 19763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:37.548966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.65:38931)
0619 20:34:37.549127 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:37.549180 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":78,"threads_started":1}
I20260619 20:34:37.645895 19764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:37.645600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.66:57677)
0619 20:34:37.645766 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:37.645829 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":81,"threads_started":1}
I20260619 20:34:37.777328 19765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:37.777008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.67:55747)
0619 20:34:37.777194 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:37.777245 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":84,"threads_started":1}
I20260619 20:34:38.550189 19766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:38.549836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.65:52087)
0619 20:34:38.550026 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:38.550093 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":86,"threads_started":1}
I20260619 20:34:38.646870 19767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:38.646553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.66:57733)
0619 20:34:38.646748 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:38.646801 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":97,"threads_started":1}
I20260619 20:34:38.778190 19768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:38.777890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.67:53717)
0619 20:34:38.778073 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:38.778123 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":88,"threads_started":1}
I20260619 20:34:39.550983 19769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:39.550724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.65:42693)
0619 20:34:39.550886 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:39.550919 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":101,"threads_started":1}
I20260619 20:34:39.647823 19770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:39.647514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.66:39145)
0619 20:34:39.647722 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:39.647759 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":97,"threads_started":1}
I20260619 20:34:39.775902 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 19295
I20260619 20:34:39.779311 19771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:39.779004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41283 (local address 127.4.83.67:58481)
0619 20:34:39.779193 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:39.779245 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":88,"threads_started":1}
I20260619 20:34:39.781687 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 19458
I20260619 20:34:39.787146 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 19592
2026-06-19T20:34:39Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18665 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:34:39 dist-test-slave-4m58 krb5kdc[19774](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:34:39 dist-test-slave-4m58 krb5kdc[19774](info): set up 2 sockets
Jun 19 20:34:39 dist-test-slave-4m58 krb5kdc[19774](info): commencing operation
krb5kdc: starting...
W20260619 20:34:40.764735 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.955s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:34:40 dist-test-slave-4m58 krb5kdc[19774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:34:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:34:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
I20260619 20:34:40.882524 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:35383
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:36871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:35383
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:34:40.962074 19787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:34:40.962304 19787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:34:40.962337 19787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:34:40.963464 19787 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 20:34:40.963503 19787 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 20:34:40.963519 19787 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 20:34:40.963534 19787 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:34:40.963549 19787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:34:40.963568 19787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:34:40.963591 19787 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:34:40.963608 19787 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:34:40.965097 19787 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:36871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:35383
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:35383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:34:40.965358 19787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:34:40.965566 19787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:34:40.971043 19793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:40.971118 19792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:40.971086 19795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:40.971161 19787 server_base.cc:1034] running on GCE node
I20260619 20:34:40.971698 19787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:34:40.972029 19787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:34:40.973193 19787 hybrid_clock.cc:648] HybridClock initialized: now 1781901280973175 us; error 36 us; skew 500 ppm
Jun 19 20:34:40 dist-test-slave-4m58 krb5kdc[19774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901280, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:34:40.975637 19787 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:34:40.976138 19787 webserver.cc:460] Webserver started at http://127.4.83.126:38505/ using document root <none> and password file <none>
I20260619 20:34:40.976351 19787 fs_manager.cc:359] Metadata directory not provided
I20260619 20:34:40.976398 19787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:34:40.976521 19787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:34:40.977598 19787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "3919dc8f8d0a478fa042332a4625c7ad"
format_stamp: "Formatted at 2026-06-19 20:34:40 on dist-test-slave-4m58"
server_key: "8b356c32d6fd560c952378cac8513f31"
server_key_iv: "dba9454d7566d8b55699f26419e09662"
server_key_version: "encryptionkey@0"
I20260619 20:34:40.977938 19787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "3919dc8f8d0a478fa042332a4625c7ad"
format_stamp: "Formatted at 2026-06-19 20:34:40 on dist-test-slave-4m58"
server_key: "8b356c32d6fd560c952378cac8513f31"
server_key_iv: "dba9454d7566d8b55699f26419e09662"
server_key_version: "encryptionkey@0"
I20260619 20:34:40.979243 19787 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:34:40.980024 19802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:40.980198 19787 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:34:40.980288 19787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "3919dc8f8d0a478fa042332a4625c7ad"
format_stamp: "Formatted at 2026-06-19 20:34:40 on dist-test-slave-4m58"
server_key: "8b356c32d6fd560c952378cac8513f31"
server_key_iv: "dba9454d7566d8b55699f26419e09662"
server_key_version: "encryptionkey@0"
I20260619 20:34:40.980351 19787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:34:40.986865 19787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:34:40.989049 19787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:34:40.989198 19787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:34:40.992987 19787 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:35383
I20260619 20:34:40.993083 19854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:35383 every 8 connection(s)
I20260619 20:34:40.993532 19787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:34:40.994189 19855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:34:40.996469 19855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad: Bootstrap starting.
I20260619 20:34:40.997032 19855 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad: Neither blocks nor log segments found. Creating new log.
I20260619 20:34:40.997300 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 19787
I20260619 20:34:40.997314 19855 log.cc:824] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad: Log is configured to *not* fsync() on all Append() calls
I20260619 20:34:40.997394 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:34:40.997575 4429 external_mini_cluster.cc:1428] Setting key a11f4618fcd77c26bf0952e0e27b151b
I20260619 20:34:40.997958 19855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad: No bootstrap required, opened a new log
I20260619 20:34:40.999729 19855 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3919dc8f8d0a478fa042332a4625c7ad" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35383 } }
I20260619 20:34:40.999899 19855 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:34:40.999938 19855 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3919dc8f8d0a478fa042332a4625c7ad, State: Initialized, Role: FOLLOWER
I20260619 20:34:41.000066 19855 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3919dc8f8d0a478fa042332a4625c7ad" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35383 } }
I20260619 20:34:41.000140 19855 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:34:41.000171 19855 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:34:41.000223 19855 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:34:41.000751 19855 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3919dc8f8d0a478fa042332a4625c7ad" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35383 } }
I20260619 20:34:41.000839 19855 leader_election.cc:302] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3919dc8f8d0a478fa042332a4625c7ad; no voters:
I20260619 20:34:41.000977 19855 leader_election.cc:288] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:34:41.001022 19860 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:34:41.001183 19860 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [term 1 LEADER]: Becoming Leader. State: Replica: 3919dc8f8d0a478fa042332a4625c7ad, State: Running, Role: LEADER
I20260619 20:34:41.001214 19855 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:34:41.001319 19860 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3919dc8f8d0a478fa042332a4625c7ad" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35383 } }
I20260619 20:34:41.001943 19861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3919dc8f8d0a478fa042332a4625c7ad. Latest consensus state: current_term: 1 leader_uuid: "3919dc8f8d0a478fa042332a4625c7ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3919dc8f8d0a478fa042332a4625c7ad" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35383 } } }
I20260619 20:34:41.002018 19861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [sys.catalog]: This master's current role is: LEADER
I20260619 20:34:41.002224 19862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3919dc8f8d0a478fa042332a4625c7ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3919dc8f8d0a478fa042332a4625c7ad" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35383 } } }
I20260619 20:34:41.002348 19862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [sys.catalog]: This master's current role is: LEADER
I20260619 20:34:41.002367 19869 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:34:41.002934 19869 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:34:41.004987 19869 catalog_manager.cc:1347] Generated new cluster ID: 1ec1df93e89c42c5a91ce4453f96bc64
I20260619 20:34:41.005043 19869 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:34:41 dist-test-slave-4m58 krb5kdc[19774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:34:41.008939 19859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:40.998085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38472 (local address 127.4.83.126:35383)
0619 20:34:40.998269 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:40.998276 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:40.998299 (+ 23us) server_negotiation.cc:407] Connection header received
0619 20:34:40.999097 (+ 798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:40.999103 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:40.999155 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:40.999281 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:40.999900 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.001356 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.002449 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.002914 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.003015 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:41.005885 (+ 2870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:41.005889 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:41.005890 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:41.005903 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:41.007483 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:41.007819 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:41.007822 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:41.007822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:41.007940 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:41.008132 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:41.008133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:41.008133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:41.008378 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:41.008519 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:41.008635 (+ 116us) server_negotiation.cc:299] Negotiation successful
0619 20:34:41.008774 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":92,"threads_started":1}
I20260619 20:34:41.038301 19869 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:34:41.038901 19869 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:34:41.045185 19869 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad: Generated new TSK 0
I20260619 20:34:41.045420 19869 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.83.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
I20260619 20:34:41.076256 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:0
--local_ip_for_outbound_sockets=127.4.83.65
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:35383
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:36871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:34:41.159230 19881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:34:41.159466 19881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:34:41.159492 19881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:34:41.160818 19881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:34:41.160924 19881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:34:41.160984 19881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:34:41.162595 19881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:36871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:35383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:34:41.162845 19881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:34:41.163076 19881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:34:41.168316 19887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:41.168357 19886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:41.168309 19881 server_base.cc:1034] running on GCE node
W20260619 20:34:41.168320 19889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:41.168772 19881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:34:41.168983 19881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:34:41.170145 19881 hybrid_clock.cc:648] HybridClock initialized: now 1781901281170136 us; error 29 us; skew 500 ppm
Jun 19 20:34:41 dist-test-slave-4m58 krb5kdc[19774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901281, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:34:41.172710 19881 init.cc:377] Logged in from keytab as kudu/127.4.83.65@KRBTEST.COM (short username kudu)
I20260619 20:34:41.173184 19881 webserver.cc:460] Webserver started at http://127.4.83.65:41743/ using document root <none> and password file <none>
I20260619 20:34:41.173398 19881 fs_manager.cc:359] Metadata directory not provided
I20260619 20:34:41.173448 19881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:34:41.173575 19881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:34:41.174634 19881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/data/instance:
uuid: "cc0e55438ef34044b5879dd1978aa101"
format_stamp: "Formatted at 2026-06-19 20:34:41 on dist-test-slave-4m58"
server_key: "22626140c19773465a6a32386c27666e"
server_key_iv: "25d4a6cf7bde51ca64a76d31cd3e5062"
server_key_version: "encryptionkey@0"
I20260619 20:34:41.174963 19881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance:
uuid: "cc0e55438ef34044b5879dd1978aa101"
format_stamp: "Formatted at 2026-06-19 20:34:41 on dist-test-slave-4m58"
server_key: "22626140c19773465a6a32386c27666e"
server_key_iv: "25d4a6cf7bde51ca64a76d31cd3e5062"
server_key_version: "encryptionkey@0"
I20260619 20:34:41.176306 19881 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:34:41.177178 19896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:41.177421 19881 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:34:41.177515 19881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "cc0e55438ef34044b5879dd1978aa101"
format_stamp: "Formatted at 2026-06-19 20:34:41 on dist-test-slave-4m58"
server_key: "22626140c19773465a6a32386c27666e"
server_key_iv: "25d4a6cf7bde51ca64a76d31cd3e5062"
server_key_version: "encryptionkey@0"
I20260619 20:34:41.177582 19881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:34:41.188659 19881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:34:41.191018 19881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:34:41.191226 19881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:34:41.191464 19881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:34:41.191778 19881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:34:41.191812 19881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:41.191872 19881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:34:41.191895 19881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:41.197654 19881 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:41803
I20260619 20:34:41.197705 20010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:41803 every 8 connection(s)
I20260619 20:34:41.198021 19881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
I20260619 20:34:41.201332 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 19881
I20260619 20:34:41.201431 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance
I20260619 20:34:41.201607 4429 external_mini_cluster.cc:1428] Setting key 08484b6aebbd596c70401812460d4c44
Jun 19 20:34:41 dist-test-slave-4m58 krb5kdc[19774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901281, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:34:41.207561 19859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:41.198731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:34231 (local address 127.4.83.126:35383)
0619 20:34:41.198778 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:41.198780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:41.199000 (+ 220us) server_negotiation.cc:407] Connection header received
0619 20:34:41.199528 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:41.199529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:41.199542 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:41.199599 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:41.200622 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.201196 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.202422 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.202621 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.202686 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:41.204985 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:41.204986 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:41.204987 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:41.204996 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:41.206281 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:41.206650 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:41.206651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:41.206652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:41.206711 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:41.206934 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:41.206935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:41.206935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:41.207049 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:41.207105 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:41.207377 (+ 272us) server_negotiation.cc:299] Negotiation successful
0619 20:34:41.207469 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:34:41.207734 20013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:41.198712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.65:34231)
0619 20:34:41.198881 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:41.198916 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:41.199423 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:41.199674 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:41.199678 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:41.200326 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:34:41.200527 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:41.200532 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.201274 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.201275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:41.202349 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:41.202351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.202688 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.202689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:41.202829 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:41.203333 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:34:41.203341 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:34:41.204895 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:34:41.206358 (+ 1463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:41.206365 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:41.206367 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:41.206585 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:41.206765 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:41.206766 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:41.206766 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:41.206879 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:41.207105 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:34:41.207111 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:34:41.207259 (+ 148us) client_negotiation.cc:769] Sending connection context
0619 20:34:41.207349 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 20:34:41.207513 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":90,"threads_started":1}
I20260619 20:34:41.208261 20011 heartbeater.cc:347] Connected to a master server at 127.4.83.126:35383
I20260619 20:34:41.208370 20011 heartbeater.cc:464] Registering TS with master...
I20260619 20:34:41.208556 20011 heartbeater.cc:511] Master 127.4.83.126:35383 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.4.83.66@KRBTEST.COM; defaulting to no policy
I20260619 20:34:41.209097 19819 ts_manager.cc:194] Registered new tserver with Master: cc0e55438ef34044b5879dd1978aa101 (127.4.83.65:41803)
I20260619 20:34:41.209746 19819 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.65@KRBTEST.COM'} at 127.4.83.65:34231
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
I20260619 20:34:41.224517 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:0
--local_ip_for_outbound_sockets=127.4.83.66
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:35383
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:36871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:34:41.306847 20016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:34:41.307073 20016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:34:41.307106 20016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:34:41.308321 20016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:34:41.308368 20016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:34:41.308403 20016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:34:41.309863 20016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:36871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:35383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:34:41.310102 20016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:34:41.310325 20016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:34:41.315788 20022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:41.315958 20021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:41.316067 20016 server_base.cc:1034] running on GCE node
W20260619 20:34:41.315789 20024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:41.316479 20016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:34:41.316761 20016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:34:41.317931 20016 hybrid_clock.cc:648] HybridClock initialized: now 1781901281317903 us; error 54 us; skew 500 ppm
Jun 19 20:34:41 dist-test-slave-4m58 krb5kdc[19774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901281, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:34:41.320825 20016 init.cc:377] Logged in from keytab as kudu/127.4.83.66@KRBTEST.COM (short username kudu)
I20260619 20:34:41.321329 20016 webserver.cc:460] Webserver started at http://127.4.83.66:43729/ using document root <none> and password file <none>
I20260619 20:34:41.321547 20016 fs_manager.cc:359] Metadata directory not provided
I20260619 20:34:41.321596 20016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:34:41.321731 20016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:34:41.322801 20016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/data/instance:
uuid: "690c4aca7db040bf8a41d8eb551dcffc"
format_stamp: "Formatted at 2026-06-19 20:34:41 on dist-test-slave-4m58"
server_key: "9f0f10da8d7af1a9ea11cb4996d9f604"
server_key_iv: "78e553f137320d382552cf0231b10239"
server_key_version: "encryptionkey@0"
I20260619 20:34:41.323158 20016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance:
uuid: "690c4aca7db040bf8a41d8eb551dcffc"
format_stamp: "Formatted at 2026-06-19 20:34:41 on dist-test-slave-4m58"
server_key: "9f0f10da8d7af1a9ea11cb4996d9f604"
server_key_iv: "78e553f137320d382552cf0231b10239"
server_key_version: "encryptionkey@0"
I20260619 20:34:41.324800 20016 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:34:41.325805 20031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:41.326148 20016 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:34:41.326294 20016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "690c4aca7db040bf8a41d8eb551dcffc"
format_stamp: "Formatted at 2026-06-19 20:34:41 on dist-test-slave-4m58"
server_key: "9f0f10da8d7af1a9ea11cb4996d9f604"
server_key_iv: "78e553f137320d382552cf0231b10239"
server_key_version: "encryptionkey@0"
I20260619 20:34:41.326373 20016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:34:41.341553 20016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:34:41.344745 20016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:34:41.344907 20016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:34:41.345361 20016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:34:41.345829 20016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:34:41.345881 20016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:41.345913 20016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:34:41.345938 20016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:41.353400 20016 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:36025
I20260619 20:34:41.353559 20145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:36025 every 8 connection(s)
I20260619 20:34:41.353806 20016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
I20260619 20:34:41.360503 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 20016
I20260619 20:34:41.360643 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance
I20260619 20:34:41.360877 4429 external_mini_cluster.cc:1428] Setting key b5253af0a750db83c03be163bcf3dc2e
Jun 19 20:34:41 dist-test-slave-4m58 krb5kdc[19774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901281, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:34:41.366981 19859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:41.354746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:50321 (local address 127.4.83.126:35383)
0619 20:34:41.354836 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:41.354839 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:41.355092 (+ 253us) server_negotiation.cc:407] Connection header received
0619 20:34:41.355690 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:41.355692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:41.355702 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:41.355775 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:41.357181 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.357694 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.359271 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.359499 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.359570 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:41.363616 (+ 4046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:41.363618 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:41.363618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:41.363628 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:41.365020 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:41.365736 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:41.365737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:41.365737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:41.365812 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:41.366195 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:41.366195 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:41.366196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:41.366321 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:41.366390 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:41.366829 (+ 439us) server_negotiation.cc:299] Negotiation successful
0619 20:34:41.366909 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 20:34:41.367336 20148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:41.354693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.66:50321)
0619 20:34:41.354901 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:41.354969 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:41.355595 (+ 626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:41.356025 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:41.356030 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:41.356752 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:34:41.357038 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:41.357045 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.357813 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.357814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:41.359074 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:41.359080 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.359610 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.359611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:41.359781 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:41.361449 (+ 1668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:34:41.361463 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:34:41.363479 (+ 2016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:34:41.365139 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:41.365154 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:41.365158 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:41.365587 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:41.365906 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:41.365907 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:41.365907 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:41.366104 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:41.366403 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:34:41.366411 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:34:41.366669 (+ 258us) client_negotiation.cc:769] Sending connection context
0619 20:34:41.366809 (+ 140us) client_negotiation.cc:240] Negotiation successful
0619 20:34:41.367039 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":99,"threads_started":1}
I20260619 20:34:41.368057 20146 heartbeater.cc:347] Connected to a master server at 127.4.83.126:35383
I20260619 20:34:41.368203 20146 heartbeater.cc:464] Registering TS with master...
I20260619 20:34:41.368497 20146 heartbeater.cc:511] Master 127.4.83.126:35383 requested a full tablet report, sending...
I20260619 20:34:41.368969 19819 ts_manager.cc:194] Registered new tserver with Master: 690c4aca7db040bf8a41d8eb551dcffc (127.4.83.66:36025)
I20260619 20:34:41.369374 19819 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.66@KRBTEST.COM'} at 127.4.83.66:50321
WARNING: no policy specified for kudu/127.4.83.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
I20260619 20:34:41.390717 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:0
--local_ip_for_outbound_sockets=127.4.83.67
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:35383
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:36871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:34:41.495631 20151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:34:41.495954 20151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:34:41.495997 20151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:34:41.497274 20151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:34:41.497319 20151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:34:41.497347 20151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:34:41.498832 20151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:36871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:35383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:34:41.499073 20151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:34:41.499297 20151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:34:41.504980 20156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:34:41.504966 20157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:41.505293 20151 server_base.cc:1034] running on GCE node
W20260619 20:34:41.504971 20159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:34:41.505604 20151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:34:41.505904 20151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:34:41.507094 20151 hybrid_clock.cc:648] HybridClock initialized: now 1781901281507076 us; error 36 us; skew 500 ppm
Jun 19 20:34:41 dist-test-slave-4m58 krb5kdc[19774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901281, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:34:41.510159 20151 init.cc:377] Logged in from keytab as kudu/127.4.83.67@KRBTEST.COM (short username kudu)
I20260619 20:34:41.510737 20151 webserver.cc:460] Webserver started at http://127.4.83.67:33341/ using document root <none> and password file <none>
I20260619 20:34:41.510967 20151 fs_manager.cc:359] Metadata directory not provided
I20260619 20:34:41.511018 20151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:34:41.511163 20151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:34:41.512526 20151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/data/instance:
uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b"
format_stamp: "Formatted at 2026-06-19 20:34:41 on dist-test-slave-4m58"
server_key: "3c99ecce406b713e497595539c44b8c1"
server_key_iv: "f8316c5232ca68e96c418270702f395c"
server_key_version: "encryptionkey@0"
I20260619 20:34:41.512926 20151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance:
uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b"
format_stamp: "Formatted at 2026-06-19 20:34:41 on dist-test-slave-4m58"
server_key: "3c99ecce406b713e497595539c44b8c1"
server_key_iv: "f8316c5232ca68e96c418270702f395c"
server_key_version: "encryptionkey@0"
I20260619 20:34:41.514489 20151 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:34:41.515416 20166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:41.515656 20151 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:34:41.515807 20151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b"
format_stamp: "Formatted at 2026-06-19 20:34:41 on dist-test-slave-4m58"
server_key: "3c99ecce406b713e497595539c44b8c1"
server_key_iv: "f8316c5232ca68e96c418270702f395c"
server_key_version: "encryptionkey@0"
I20260619 20:34:41.515990 20151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:34:41.536195 20151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:34:41.538311 20151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:34:41.538437 20151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:34:41.538640 20151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:34:41.538990 20151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:34:41.539033 20151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:41.539057 20151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:34:41.539069 20151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:34:41.546011 20151 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:41165
I20260619 20:34:41.546111 20282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:41165 every 8 connection(s)
I20260619 20:34:41.546429 20151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
I20260619 20:34:41.547032 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 20151
I20260619 20:34:41.547132 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance
I20260619 20:34:41.547354 4429 external_mini_cluster.cc:1428] Setting key 16b3c6e46a415b14635fbf79b66e92eb
Jun 19 20:34:41 dist-test-slave-4m58 krb5kdc[19774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901281, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:34:41.558069 19859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:41.547329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:47259 (local address 127.4.83.126:35383)
0619 20:34:41.547401 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:41.547404 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:41.547735 (+ 331us) server_negotiation.cc:407] Connection header received
0619 20:34:41.548471 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:41.548472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:41.548487 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:41.548532 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:41.549986 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.550552 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.551683 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.551964 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.552039 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:41.554795 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:41.554797 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:41.554797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:41.554836 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:41.556395 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:41.556955 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:41.556956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:41.556956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:41.557018 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:41.557325 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:41.557326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:41.557326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:41.557476 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:41.557540 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:41.557853 (+ 313us) server_negotiation.cc:299] Negotiation successful
0619 20:34:41.557956 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 20:34:41.558531 20285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:41.547319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.67:47259)
0619 20:34:41.547555 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:41.547621 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:41.548362 (+ 741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:41.548639 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:41.548643 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:41.549512 (+ 869us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:34:41.549848 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:41.549856 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.550655 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.550657 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:41.551577 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:41.551580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.552046 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.552047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:41.552200 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:41.552860 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:34:41.552870 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:34:41.554658 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:34:41.556522 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:41.556540 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:41.556543 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:41.556865 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:41.557116 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:34:41.557117 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:34:41.557117 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:34:41.557249 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:34:41.557538 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:34:41.557547 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:34:41.557721 (+ 174us) client_negotiation.cc:769] Sending connection context
0619 20:34:41.557827 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 20:34:41.558038 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":101,"threads_started":1}
I20260619 20:34:41.559132 20283 heartbeater.cc:347] Connected to a master server at 127.4.83.126:35383
I20260619 20:34:41.559266 20283 heartbeater.cc:464] Registering TS with master...
I20260619 20:34:41.559537 20283 heartbeater.cc:511] Master 127.4.83.126:35383 requested a full tablet report, sending...
I20260619 20:34:41.560029 19819 ts_manager.cc:194] Registered new tserver with Master: e505f8b1bfb04b5f9d4d6e0c54fb4d1b (127.4.83.67:41165)
I20260619 20:34:41.560467 19819 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.67@KRBTEST.COM'} at 127.4.83.67:47259
I20260619 20:34:41.570690 4429 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:34:41.579231 19859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:41.572205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38484 (local address 127.4.83.126:35383)
0619 20:34:41.572285 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:41.572288 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:41.572389 (+ 101us) server_negotiation.cc:407] Connection header received
0619 20:34:41.572648 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:41.572649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:41.572665 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:41.572718 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:34:41.573394 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.573995 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.574798 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.574996 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.575059 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:41.576571 (+ 1512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:34:41.576573 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:34:41.576574 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:34:41.576585 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:34:41.577976 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:41.578343 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:41.578344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:41.578345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:41.578409 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:34:41.578576 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:34:41.578577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:34:41.578577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:34:41.578790 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:34:41.578889 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:41.579017 (+ 128us) server_negotiation.cc:299] Negotiation successful
0619 20:34:41.579117 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 20:34:41.581359 19819 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38484:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:34:41.583374 19819 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:34:41.590895 20294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:41.586751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:41165 (local address 127.0.0.1:44622)
0619 20:34:41.586942 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:41.586999 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:41.587083 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:41.587682 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:41.587685 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:41.587689 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:34:41.587780 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:41.587783 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.588820 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.588821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:41.589910 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:41.589913 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.590566 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.590567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:41.590655 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:41.590669 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 20:34:41.590743 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 20:34:41.590812 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":93,"threads_started":1}
I20260619 20:34:41.590903 20299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:41.587267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44622 (local address 127.4.83.67:41165)
0619 20:34:41.587494 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:41.587498 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:41.587508 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:34:41.587546 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:41.587548 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:41.587564 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:41.587882 (+ 318us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:34:41.587965 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.588730 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.589983 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.590491 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.590554 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:41.590615 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:41.590763 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 20:34:41.590806 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":120,"threads_started":1}
I20260619 20:34:41.591575 20216 tablet_service.cc:1467] Processing CreateTablet for tablet c844af98e96c49d3ab712f971eb83f28 (DEFAULT_TABLE table=test-table [id=502cebb10dc24d738c8db5c987269059]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:34:41.591946 20216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c844af98e96c49d3ab712f971eb83f28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:34:41.592172 20295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:41.586751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:36025 (local address 127.0.0.1:33544)
0619 20:34:41.586991 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:41.587004 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:41.587084 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:41.587703 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:41.587703 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:41.587705 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:34:41.587786 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:41.587787 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.589496 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.589497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:41.590747 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:41.590750 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.591825 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.591826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:41.591935 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:41.591940 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:34:41.592000 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 20:34:41.592062 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":50,"thread_start_us":108,"threads_started":1}
I20260619 20:34:41.592484 20296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:41.586899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33544 (local address 127.4.83.66:36025)
0619 20:34:41.587445 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:41.587450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:41.587464 (+ 14us) server_negotiation.cc:407] Connection header received
0619 20:34:41.587510 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:41.587513 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:41.587530 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:41.587617 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:34:41.587934 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.589260 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.590854 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.591650 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.591722 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:41.591786 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:41.592334 (+ 548us) server_negotiation.cc:299] Negotiation successful
0619 20:34:41.592395 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":132,"threads_started":1}
I20260619 20:34:41.592531 20297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:41.587040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:41803 (local address 127.0.0.1:60376)
0619 20:34:41.587510 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:41.587520 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:41.587575 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:41.588003 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:41.588004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:41.588006 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:34:41.588044 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:41.588046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.589409 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.589410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:41.590748 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:41.590750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.592186 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.592187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:41.592244 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:41.592247 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:34:41.592406 (+ 159us) client_negotiation.cc:240] Negotiation successful
0619 20:34:41.592467 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":37,"thread_start_us":111,"threads_started":1}
I20260619 20:34:41.592773 20298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:41.587210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60376 (local address 127.4.83.65:41803)
0619 20:34:41.587789 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:41.587795 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:41.587805 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:34:41.587873 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:41.587876 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:41.587894 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:41.587990 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:34:41.588131 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.589256 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.590855 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.592033 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.592090 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:41.592142 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:41.592637 (+ 495us) server_negotiation.cc:299] Negotiation successful
0619 20:34:41.592689 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":108,"threads_started":1}
I20260619 20:34:41.593353 19945 tablet_service.cc:1467] Processing CreateTablet for tablet c844af98e96c49d3ab712f971eb83f28 (DEFAULT_TABLE table=test-table [id=502cebb10dc24d738c8db5c987269059]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:34:41.593353 20080 tablet_service.cc:1467] Processing CreateTablet for tablet c844af98e96c49d3ab712f971eb83f28 (DEFAULT_TABLE table=test-table [id=502cebb10dc24d738c8db5c987269059]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:34:41.593750 20080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c844af98e96c49d3ab712f971eb83f28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:34:41.593750 19945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c844af98e96c49d3ab712f971eb83f28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:34:41.595141 20300 tablet_bootstrap.cc:492] T c844af98e96c49d3ab712f971eb83f28 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b: Bootstrap starting.
I20260619 20:34:41.595785 20300 tablet_bootstrap.cc:654] T c844af98e96c49d3ab712f971eb83f28 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b: Neither blocks nor log segments found. Creating new log.
I20260619 20:34:41.596282 20300 log.cc:824] T c844af98e96c49d3ab712f971eb83f28 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b: Log is configured to *not* fsync() on all Append() calls
I20260619 20:34:41.596936 20300 tablet_bootstrap.cc:492] T c844af98e96c49d3ab712f971eb83f28 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b: No bootstrap required, opened a new log
I20260619 20:34:41.596998 20300 ts_tablet_manager.cc:1397] T c844af98e96c49d3ab712f971eb83f28 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:34:41.597530 20302 tablet_bootstrap.cc:492] T c844af98e96c49d3ab712f971eb83f28 P cc0e55438ef34044b5879dd1978aa101: Bootstrap starting.
I20260619 20:34:41.598342 20302 tablet_bootstrap.cc:654] T c844af98e96c49d3ab712f971eb83f28 P cc0e55438ef34044b5879dd1978aa101: Neither blocks nor log segments found. Creating new log.
I20260619 20:34:41.598907 20302 log.cc:824] T c844af98e96c49d3ab712f971eb83f28 P cc0e55438ef34044b5879dd1978aa101: Log is configured to *not* fsync() on all Append() calls
I20260619 20:34:41.598997 20300 raft_consensus.cc:348] T c844af98e96c49d3ab712f971eb83f28 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41165 } } peers { permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 } } peers { permanent_uuid: "690c4aca7db040bf8a41d8eb551dcffc" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 36025 } }
I20260619 20:34:41.599195 20300 raft_consensus.cc:374] T c844af98e96c49d3ab712f971eb83f28 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:34:41.599233 20300 raft_consensus.cc:729] T c844af98e96c49d3ab712f971eb83f28 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e505f8b1bfb04b5f9d4d6e0c54fb4d1b, State: Initialized, Role: FOLLOWER
I20260619 20:34:41.599334 20303 tablet_bootstrap.cc:492] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc: Bootstrap starting.
I20260619 20:34:41.599387 20300 consensus_queue.cc:260] T c844af98e96c49d3ab712f971eb83f28 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41165 } } peers { permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 } } peers { permanent_uuid: "690c4aca7db040bf8a41d8eb551dcffc" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 36025 } }
I20260619 20:34:41.599663 20300 ts_tablet_manager.cc:1428] T c844af98e96c49d3ab712f971eb83f28 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:34:41.599862 20302 tablet_bootstrap.cc:492] T c844af98e96c49d3ab712f971eb83f28 P cc0e55438ef34044b5879dd1978aa101: No bootstrap required, opened a new log
I20260619 20:34:41.599927 20283 heartbeater.cc:503] Master 127.4.83.126:35383 was elected leader, sending a full tablet report...
I20260619 20:34:41.599959 20302 ts_tablet_manager.cc:1397] T c844af98e96c49d3ab712f971eb83f28 P cc0e55438ef34044b5879dd1978aa101: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 20:34:41.600021 20303 tablet_bootstrap.cc:654] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc: Neither blocks nor log segments found. Creating new log.
I20260619 20:34:41.600487 20303 log.cc:824] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc: Log is configured to *not* fsync() on all Append() calls
I20260619 20:34:41.601264 20303 tablet_bootstrap.cc:492] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc: No bootstrap required, opened a new log
I20260619 20:34:41.601332 20303 ts_tablet_manager.cc:1397] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:34:41.602407 20302 raft_consensus.cc:348] T c844af98e96c49d3ab712f971eb83f28 P cc0e55438ef34044b5879dd1978aa101 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41165 } } peers { permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 } } peers { permanent_uuid: "690c4aca7db040bf8a41d8eb551dcffc" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 36025 } }
I20260619 20:34:41.602588 20302 raft_consensus.cc:374] T c844af98e96c49d3ab712f971eb83f28 P cc0e55438ef34044b5879dd1978aa101 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:34:41.602612 20302 raft_consensus.cc:729] T c844af98e96c49d3ab712f971eb83f28 P cc0e55438ef34044b5879dd1978aa101 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc0e55438ef34044b5879dd1978aa101, State: Initialized, Role: FOLLOWER
I20260619 20:34:41.602730 20302 consensus_queue.cc:260] T c844af98e96c49d3ab712f971eb83f28 P cc0e55438ef34044b5879dd1978aa101 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41165 } } peers { permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 } } peers { permanent_uuid: "690c4aca7db040bf8a41d8eb551dcffc" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 36025 } }
I20260619 20:34:41.603063 20302 ts_tablet_manager.cc:1428] T c844af98e96c49d3ab712f971eb83f28 P cc0e55438ef34044b5879dd1978aa101: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:34:41.603150 20011 heartbeater.cc:503] Master 127.4.83.126:35383 was elected leader, sending a full tablet report...
I20260619 20:34:41.603060 20303 raft_consensus.cc:348] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41165 } } peers { permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 } } peers { permanent_uuid: "690c4aca7db040bf8a41d8eb551dcffc" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 36025 } }
I20260619 20:34:41.603262 20303 raft_consensus.cc:374] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:34:41.603307 20303 raft_consensus.cc:729] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 690c4aca7db040bf8a41d8eb551dcffc, State: Initialized, Role: FOLLOWER
I20260619 20:34:41.603430 20303 consensus_queue.cc:260] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41165 } } peers { permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 } } peers { permanent_uuid: "690c4aca7db040bf8a41d8eb551dcffc" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 36025 } }
I20260619 20:34:41.603713 20303 ts_tablet_manager.cc:1428] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 20:34:41.604053 20146 heartbeater.cc:503] Master 127.4.83.126:35383 was elected leader, sending a full tablet report...
W20260619 20:34:41.604444 20147 tablet.cc:2369] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:34:41.617242 20308 raft_consensus.cc:482] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:34:41.617389 20308 raft_consensus.cc:504] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41165 } } peers { permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 } } peers { permanent_uuid: "690c4aca7db040bf8a41d8eb551dcffc" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 36025 } }
I20260619 20:34:41.617763 20308 leader_election.cc:288] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e505f8b1bfb04b5f9d4d6e0c54fb4d1b (127.4.83.67:41165), cc0e55438ef34044b5879dd1978aa101 (127.4.83.65:41803)
I20260619 20:34:41.622372 20299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:41.618090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:44473 (local address 127.4.83.67:41165)
0619 20:34:41.618190 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:41.618192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:41.618716 (+ 524us) server_negotiation.cc:407] Connection header received
0619 20:34:41.618749 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:41.618750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:41.618763 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:41.618815 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:34:41.619072 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.619990 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.621388 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.621987 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.622048 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:41.622057 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:41.622253 (+ 196us) server_negotiation.cc:299] Negotiation successful
0619 20:34:41.622291 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260619 20:34:41.622398 20298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:41.618341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:36007 (local address 127.4.83.65:41803)
0619 20:34:41.618390 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:41.618393 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:41.618493 (+ 100us) server_negotiation.cc:407] Connection header received
0619 20:34:41.618618 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:41.618620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:41.618656 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:41.618688 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:34:41.619013 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.619892 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.621285 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.621864 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.621946 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:41.621960 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:41.622235 (+ 275us) server_negotiation.cc:299] Negotiation successful
0619 20:34:41.622281 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 20:34:41.622434 20148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:41.617992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:41165 (local address 127.4.83.66:44473)
0619 20:34:41.618056 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:41.618071 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:41.618655 (+ 584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:41.618856 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:41.618857 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:41.618859 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:34:41.618967 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:41.618968 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.620107 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.620108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:41.621275 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:41.621279 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.622054 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.622055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:41.622155 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:41.622160 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:34:41.622213 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 20:34:41.622317 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":19}
I20260619 20:34:41.623014 20236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c844af98e96c49d3ab712f971eb83f28" candidate_uuid: "690c4aca7db040bf8a41d8eb551dcffc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" is_pre_election: true
I20260619 20:34:41.623212 20236 raft_consensus.cc:2454] T c844af98e96c49d3ab712f971eb83f28 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 690c4aca7db040bf8a41d8eb551dcffc in term 0.
I20260619 20:34:41.623538 20035 leader_election.cc:302] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 690c4aca7db040bf8a41d8eb551dcffc, e505f8b1bfb04b5f9d4d6e0c54fb4d1b; no voters:
I20260619 20:34:41.623703 20308 raft_consensus.cc:2781] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:34:41.623728 20309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:41.618142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:41803 (local address 127.4.83.66:36007)
0619 20:34:41.618398 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:41.618418 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 20:34:41.618534 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:34:41.618793 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:34:41.618794 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:34:41.618796 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:34:41.618899 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:41.618901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.619988 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.619990 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:41.621195 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:34:41.621199 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.621981 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.621983 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:34:41.622118 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:41.622127 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 20:34:41.622198 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 20:34:41.623233 (+ 1035us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":1,"thread_start_us":186,"threads_started":1}
I20260619 20:34:41.623759 20308 raft_consensus.cc:482] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:34:41.623994 20308 raft_consensus.cc:3037] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:34:41.624625 19965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c844af98e96c49d3ab712f971eb83f28" candidate_uuid: "690c4aca7db040bf8a41d8eb551dcffc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc0e55438ef34044b5879dd1978aa101" is_pre_election: true
I20260619 20:34:41.624827 19965 raft_consensus.cc:2454] T c844af98e96c49d3ab712f971eb83f28 P cc0e55438ef34044b5879dd1978aa101 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 690c4aca7db040bf8a41d8eb551dcffc in term 0.
I20260619 20:34:41.624971 20308 raft_consensus.cc:504] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41165 } } peers { permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 } } peers { permanent_uuid: "690c4aca7db040bf8a41d8eb551dcffc" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 36025 } }
I20260619 20:34:41.625123 20308 leader_election.cc:288] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc [CANDIDATE]: Term 1 election: Requested vote from peers e505f8b1bfb04b5f9d4d6e0c54fb4d1b (127.4.83.67:41165), cc0e55438ef34044b5879dd1978aa101 (127.4.83.65:41803)
I20260619 20:34:41.625330 20236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c844af98e96c49d3ab712f971eb83f28" candidate_uuid: "690c4aca7db040bf8a41d8eb551dcffc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b"
I20260619 20:34:41.625339 19965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c844af98e96c49d3ab712f971eb83f28" candidate_uuid: "690c4aca7db040bf8a41d8eb551dcffc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc0e55438ef34044b5879dd1978aa101"
I20260619 20:34:41.625402 20236 raft_consensus.cc:3037] T c844af98e96c49d3ab712f971eb83f28 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:34:41.625416 19965 raft_consensus.cc:3037] T c844af98e96c49d3ab712f971eb83f28 P cc0e55438ef34044b5879dd1978aa101 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:34:41.626174 20236 raft_consensus.cc:2454] T c844af98e96c49d3ab712f971eb83f28 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 690c4aca7db040bf8a41d8eb551dcffc in term 1.
I20260619 20:34:41.626338 20035 leader_election.cc:302] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 690c4aca7db040bf8a41d8eb551dcffc, e505f8b1bfb04b5f9d4d6e0c54fb4d1b; no voters:
I20260619 20:34:41.626474 20308 raft_consensus.cc:2781] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:34:41.626541 19965 raft_consensus.cc:2454] T c844af98e96c49d3ab712f971eb83f28 P cc0e55438ef34044b5879dd1978aa101 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 690c4aca7db040bf8a41d8eb551dcffc in term 1.
I20260619 20:34:41.626657 20308 raft_consensus.cc:686] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc [term 1 LEADER]: Becoming Leader. State: Replica: 690c4aca7db040bf8a41d8eb551dcffc, State: Running, Role: LEADER
I20260619 20:34:41.626773 20308 consensus_queue.cc:237] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41165 } } peers { permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 } } peers { permanent_uuid: "690c4aca7db040bf8a41d8eb551dcffc" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 36025 } }
I20260619 20:34:41.627707 19819 catalog_manager.cc:5495] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc reported cstate change: term changed from 0 to 1, leader changed from <none> to 690c4aca7db040bf8a41d8eb551dcffc (127.4.83.66). New cstate: current_term: 1 leader_uuid: "690c4aca7db040bf8a41d8eb551dcffc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "690c4aca7db040bf8a41d8eb551dcffc" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 36025 } health_report { overall_health: HEALTHY } } }
I20260619 20:34:41.641067 20296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:41.638507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33548 (local address 127.4.83.66:36025)
0619 20:34:41.638599 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0619 20:34:41.638601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:34:41.638609 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:34:41.638656 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:34:41.638657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:34:41.638686 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:34:41.638754 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:34:41.638906 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.639431 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.640275 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:34:41.640478 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:34:41.640545 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:34:41.640704 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:34:41.640840 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:34:41.640921 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 20:34:41.640938 (+ 17us) server_negotiation.cc:299] Negotiation successful
0619 20:34:41.640963 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 20:34:41.643390 20236 raft_consensus.cc:1261] T c844af98e96c49d3ab712f971eb83f28 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [term 1 FOLLOWER]: Refusing update from remote peer 690c4aca7db040bf8a41d8eb551dcffc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:34:41.643417 19965 raft_consensus.cc:1261] T c844af98e96c49d3ab712f971eb83f28 P cc0e55438ef34044b5879dd1978aa101 [term 1 FOLLOWER]: Refusing update from remote peer 690c4aca7db040bf8a41d8eb551dcffc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:34:41.643813 20310 consensus_queue.cc:1035] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:34:41.644030 20308 consensus_queue.cc:1035] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:34:41.646775 20313 mvcc.cc:205] Tried to move back new op lower bound from 7298667649607991296 to 7298667649543946240. Current Snapshot: MvccSnapshot[applied={T|T < 7298667649607991296}]
I20260619 20:34:41.647266 20314 mvcc.cc:205] Tried to move back new op lower bound from 7298667649607991296 to 7298667649543946240. Current Snapshot: MvccSnapshot[applied={T|T < 7298667649607991296}]
I20260619 20:34:41.650357 19818 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:34:41.650482 19818 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:34:41.651288 19818 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad: Sending DeleteTablet for 3 replicas of tablet c844af98e96c49d3ab712f971eb83f28
I20260619 20:34:41.651702 20216 tablet_service.cc:1514] Processing DeleteTablet for tablet c844af98e96c49d3ab712f971eb83f28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:34:41 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:44622
I20260619 20:34:41.651710 19945 tablet_service.cc:1514] Processing DeleteTablet for tablet c844af98e96c49d3ab712f971eb83f28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:34:41 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:60376
I20260619 20:34:41.651760 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 19881
I20260619 20:34:41.652601 20080 tablet_service.cc:1514] Processing DeleteTablet for tablet c844af98e96c49d3ab712f971eb83f28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:34:41 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:33544
I20260619 20:34:41.652860 20324 tablet_replica.cc:330] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc: stopping tablet replica
I20260619 20:34:41.652961 20324 raft_consensus.cc:2229] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:34:41.653069 20324 raft_consensus.cc:2258] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:34:41.653625 20324 ts_tablet_manager.cc:1905] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:34:41.655453 20324 ts_tablet_manager.cc:1918] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:34:41.655539 20324 log.cc:1196] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/wal/wals/c844af98e96c49d3ab712f971eb83f28
I20260619 20:34:41.655807 20324 ts_tablet_manager.cc:1939] T c844af98e96c49d3ab712f971eb83f28 P 690c4aca7db040bf8a41d8eb551dcffc: Deleting consensus metadata
I20260619 20:34:41.656296 19805 catalog_manager.cc:4841] TS 690c4aca7db040bf8a41d8eb551dcffc (127.4.83.66:36025): tablet c844af98e96c49d3ab712f971eb83f28 (table test-table [id=502cebb10dc24d738c8db5c987269059]) successfully deleted
W20260619 20:34:41.658591 19806 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.83.65:41803 (error 108)
I20260619 20:34:41.658867 20323 tablet_replica.cc:330] T c844af98e96c49d3ab712f971eb83f28 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b: stopping tablet replica
I20260619 20:34:41.659047 20323 raft_consensus.cc:2229] T c844af98e96c49d3ab712f971eb83f28 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:34:41.659190 20323 raft_consensus.cc:2258] T c844af98e96c49d3ab712f971eb83f28 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:34:41.659193 20297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:34:41.659068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:41803 (local address 127.0.0.1:60378)
0619 20:34:41.659116 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 20:34:41.659154 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.65:41803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260619 20:34:41.659314 19806 catalog_manager.cc:4584] TS cc0e55438ef34044b5879dd1978aa101 (127.4.83.65:41803): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c844af98e96c49d3ab712f971eb83f28: Network error: Client connection negotiation failed: client connection to 127.4.83.65:41803: connect: Connection refused (error 111)
I20260619 20:34:41.659436 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 20016
I20260619 20:34:41.660634 20323 ts_tablet_manager.cc:1905] T c844af98e96c49d3ab712f971eb83f28 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:34:41.661998 20323 ts_tablet_manager.cc:1918] T c844af98e96c49d3ab712f971eb83f28 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:34:41.662073 20323 log.cc:1196] T c844af98e96c49d3ab712f971eb83f28 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/wal/wals/c844af98e96c49d3ab712f971eb83f28
I20260619 20:34:41.662257 20323 ts_tablet_manager.cc:1939] T c844af98e96c49d3ab712f971eb83f28 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b: Deleting consensus metadata
I20260619 20:34:41.662755 19806 catalog_manager.cc:4841] TS e505f8b1bfb04b5f9d4d6e0c54fb4d1b (127.4.83.67:41165): tablet c844af98e96c49d3ab712f971eb83f28 (table test-table [id=502cebb10dc24d738c8db5c987269059]) successfully deleted
I20260619 20:34:41.665146 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 20151
I20260619 20:34:41.669708 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 19787
I20260619 20:35:01.675663 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:35383
--webserver_interface=127.4.83.126
--webserver_port=38505
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:36871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:35383
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:35:01.764982 20326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:01.765208 20326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:01.765242 20326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:01.766403 20326 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 20:35:01.766450 20326 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 20:35:01.766467 20326 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 20:35:01.766482 20326 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:35:01.766497 20326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:01.766513 20326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:01.766536 20326 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:35:01.766556 20326 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:35:01.768111 20326 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:36871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:35383
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:35383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=38505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:01.768368 20326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:01.768597 20326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:35:01.777069 20326 server_base.cc:1034] running on GCE node
W20260619 20:35:01.777005 20335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:01.777025 20333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:01.777043 20332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:01.777617 20326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:01.777948 20326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:01.779104 20326 hybrid_clock.cc:648] HybridClock initialized: now 1781901301779089 us; error 32 us; skew 500 ppm
Jun 19 20:35:01 dist-test-slave-4m58 krb5kdc[19774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901301, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:35:01.781574 20326 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:35:01.782019 20326 webserver.cc:460] Webserver started at http://127.4.83.126:38505/ using document root <none> and password file <none>
I20260619 20:35:01.782228 20326 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:01.782291 20326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:01.783612 20326 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:01.784296 20342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:01.784487 20326 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:35:01.784570 20326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "3919dc8f8d0a478fa042332a4625c7ad"
format_stamp: "Formatted at 2026-06-19 20:34:40 on dist-test-slave-4m58"
server_key: "8b356c32d6fd560c952378cac8513f31"
server_key_iv: "dba9454d7566d8b55699f26419e09662"
server_key_version: "encryptionkey@0"
I20260619 20:35:01.784974 20326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:01.801004 20326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:01.803114 20326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:01.803254 20326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:01.807533 20326 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:35383
I20260619 20:35:01.807657 20395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:35383 every 8 connection(s)
I20260619 20:35:01.808060 20326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:35:01.809650 20396 sys_catalog.cc:263] Verifying existing consensus state
I20260619 20:35:01.810271 20396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad: Bootstrap starting.
I20260619 20:35:01.813180 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 20326
I20260619 20:35:01.813462 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:41803
--local_ip_for_outbound_sockets=127.4.83.65
--tserver_master_addrs=127.4.83.126:35383
--webserver_port=41743
--webserver_interface=127.4.83.65
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:36871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:35:01.818585 20396 log.cc:824] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:01.820756 20396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:35:01.821071 20396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad: Bootstrap complete.
I20260619 20:35:01.823143 20396 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3919dc8f8d0a478fa042332a4625c7ad" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35383 } }
I20260619 20:35:01.823388 20396 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3919dc8f8d0a478fa042332a4625c7ad, State: Initialized, Role: FOLLOWER
I20260619 20:35:01.823545 20396 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3919dc8f8d0a478fa042332a4625c7ad" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35383 } }
I20260619 20:35:01.823606 20396 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:35:01.823649 20396 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:35:01.823695 20396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [term 1 FOLLOWER]: Advancing to term 2
I20260619 20:35:01.824853 20396 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3919dc8f8d0a478fa042332a4625c7ad" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35383 } }
I20260619 20:35:01.824952 20396 leader_election.cc:302] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3919dc8f8d0a478fa042332a4625c7ad; no voters:
I20260619 20:35:01.825140 20396 leader_election.cc:288] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 20:35:01.825201 20400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [term 2 FOLLOWER]: Leader election won for term 2
I20260619 20:35:01.825354 20400 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [term 2 LEADER]: Becoming Leader. State: Replica: 3919dc8f8d0a478fa042332a4625c7ad, State: Running, Role: LEADER
I20260619 20:35:01.825397 20396 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:35:01.825449 20400 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3919dc8f8d0a478fa042332a4625c7ad" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35383 } }
I20260619 20:35:01.825742 20401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3919dc8f8d0a478fa042332a4625c7ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3919dc8f8d0a478fa042332a4625c7ad" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35383 } } }
I20260619 20:35:01.825819 20401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:01.826066 20402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3919dc8f8d0a478fa042332a4625c7ad. Latest consensus state: current_term: 2 leader_uuid: "3919dc8f8d0a478fa042332a4625c7ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3919dc8f8d0a478fa042332a4625c7ad" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35383 } } }
I20260619 20:35:01.826125 20405 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:35:01.827088 20405 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:35:01.827236 20405 catalog_manager.cc:1259] Loaded cluster ID: 1ec1df93e89c42c5a91ce4453f96bc64
I20260619 20:35:01.827517 20405 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:35:01.827270 20402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:01.828701 20405 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:35:01.828958 20405 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad: Loaded TSK: 0
I20260619 20:35:01.838963 20405 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad: Generated new TSK 1
I20260619 20:35:01.839088 20405 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 20:35:01.897476 20398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:01.897718 20398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:01.897753 20398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:01.898967 20398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:01.899013 20398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:01.899041 20398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:35:01.900624 20398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:36871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:41803
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=41743
--tserver_master_addrs=127.4.83.126:35383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:01.900872 20398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:01.901109 20398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:01.907290 20423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:01.907430 20398 server_base.cc:1034] running on GCE node
W20260619 20:35:01.907271 20426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:01.907271 20424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:01.907707 20398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:01.907969 20398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:01.909122 20398 hybrid_clock.cc:648] HybridClock initialized: now 1781901301909111 us; error 32 us; skew 500 ppm
Jun 19 20:35:01 dist-test-slave-4m58 krb5kdc[19774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901301, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:35:01.911741 20398 init.cc:377] Logged in from keytab as kudu/127.4.83.65@KRBTEST.COM (short username kudu)
I20260619 20:35:01.912240 20398 webserver.cc:460] Webserver started at http://127.4.83.65:41743/ using document root <none> and password file <none>
I20260619 20:35:01.912472 20398 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:01.912544 20398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:01.913885 20398 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:35:01.914505 20433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:01.914697 20398 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:35:01.914805 20398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "cc0e55438ef34044b5879dd1978aa101"
format_stamp: "Formatted at 2026-06-19 20:34:41 on dist-test-slave-4m58"
server_key: "22626140c19773465a6a32386c27666e"
server_key_iv: "25d4a6cf7bde51ca64a76d31cd3e5062"
server_key_version: "encryptionkey@0"
I20260619 20:35:01.915282 20398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:01.942005 20398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:01.944823 20398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:01.944998 20398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:01.945274 20398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:35:01.945832 20440 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 20:35:01.947290 20398 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 20:35:01.947352 20398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 20:35:01.947389 20398 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 20:35:01.948199 20398 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 20:35:01.948235 20398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:01.948334 20440 tablet_bootstrap.cc:492] T c844af98e96c49d3ab712f971eb83f28 P cc0e55438ef34044b5879dd1978aa101: Bootstrap starting.
I20260619 20:35:01.954905 20398 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:41803
I20260619 20:35:01.955051 20548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:41803 every 8 connection(s)
I20260619 20:35:01.955324 20398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
I20260619 20:35:01.958688 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 20398
I20260619 20:35:01.959096 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:36025
--local_ip_for_outbound_sockets=127.4.83.66
--tserver_master_addrs=127.4.83.126:35383
--webserver_port=43729
--webserver_interface=127.4.83.66
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:36871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:35:01.959921 20440 log.cc:824] T c844af98e96c49d3ab712f971eb83f28 P cc0e55438ef34044b5879dd1978aa101: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:01.962931 20440 tablet_bootstrap.cc:492] T c844af98e96c49d3ab712f971eb83f28 P cc0e55438ef34044b5879dd1978aa101: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:35:01.963267 20440 tablet_bootstrap.cc:492] T c844af98e96c49d3ab712f971eb83f28 P cc0e55438ef34044b5879dd1978aa101: Bootstrap complete.
Jun 19 20:35:01 dist-test-slave-4m58 krb5kdc[19774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901301, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:35:01.964433 20440 ts_tablet_manager.cc:1397] T c844af98e96c49d3ab712f971eb83f28 P cc0e55438ef34044b5879dd1978aa101: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.014s
I20260619 20:35:01.966377 20440 raft_consensus.cc:348] T c844af98e96c49d3ab712f971eb83f28 P cc0e55438ef34044b5879dd1978aa101 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41165 } } peers { permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 } } peers { permanent_uuid: "690c4aca7db040bf8a41d8eb551dcffc" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 36025 } }
I20260619 20:35:01.966710 20440 raft_consensus.cc:729] T c844af98e96c49d3ab712f971eb83f28 P cc0e55438ef34044b5879dd1978aa101 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc0e55438ef34044b5879dd1978aa101, State: Initialized, Role: FOLLOWER
I20260619 20:35:01.966876 20440 consensus_queue.cc:260] T c844af98e96c49d3ab712f971eb83f28 P cc0e55438ef34044b5879dd1978aa101 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41165 } } peers { permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 } } peers { permanent_uuid: "690c4aca7db040bf8a41d8eb551dcffc" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 36025 } }
I20260619 20:35:01.967185 20440 ts_tablet_manager.cc:1428] T c844af98e96c49d3ab712f971eb83f28 P cc0e55438ef34044b5879dd1978aa101: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 20:35:01.968376 20551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:01.956018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.65:32943)
0619 20:35:01.956183 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:01.956226 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:01.956876 (+ 650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:01.957326 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:01.957330 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:01.957622 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:35:01.957812 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:01.957818 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:01.959568 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:01.959569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:01.960846 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:01.960850 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:01.961666 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:01.961668 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:01.961828 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:01.962454 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:35:01.962463 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:35:01.964098 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:35:01.966299 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:35:01.966307 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:35:01.966310 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:35:01.966618 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:35:01.966925 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:35:01.966926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:35:01.966926 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:35:01.967125 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:35:01.967688 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:35:01.967692 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:35:01.967866 (+ 174us) client_negotiation.cc:769] Sending connection context
0619 20:35:01.967986 (+ 120us) client_negotiation.cc:240] Negotiation successful
0619 20:35:01.968175 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260619 20:35:01.968437 20552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:01.956125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:32943 (local address 127.4.83.126:35383)
0619 20:35:01.956314 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:01.956322 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:01.956367 (+ 45us) server_negotiation.cc:407] Connection header received
0619 20:35:01.957124 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:01.957129 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:01.957173 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:01.957287 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:01.957928 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:01.959413 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:01.960932 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:01.961539 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:01.962133 (+ 594us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:01.964216 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:01.964220 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:01.964221 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:35:01.964233 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:01.966189 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:01.966723 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:01.966727 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:01.966727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:01.966845 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:01.967235 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:01.967236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:01.967236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:01.967598 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:01.967694 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:01.968072 (+ 378us) server_negotiation.cc:299] Negotiation successful
0619 20:35:01.968225 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":84,"threads_started":1}
I20260619 20:35:01.969187 20549 heartbeater.cc:347] Connected to a master server at 127.4.83.126:35383
I20260619 20:35:01.969288 20549 heartbeater.cc:464] Registering TS with master...
I20260619 20:35:01.969481 20549 heartbeater.cc:511] Master 127.4.83.126:35383 requested a full tablet report, sending...
I20260619 20:35:01.970242 20359 ts_manager.cc:194] Registered new tserver with Master: cc0e55438ef34044b5879dd1978aa101 (127.4.83.65:41803)
I20260619 20:35:01.971235 20359 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.65@KRBTEST.COM'} at 127.4.83.65:32943
I20260619 20:35:01.972280 20549 heartbeater.cc:503] Master 127.4.83.126:35383 was elected leader, sending a full tablet report...
I20260619 20:35:01.977955 20557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:01.970879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50392 (local address 127.4.83.65:41803)
0619 20:35:01.974782 (+ 3903us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:01.974786 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:01.974978 (+ 192us) server_negotiation.cc:407] Connection header received
0619 20:35:01.975076 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:01.975079 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:01.975095 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:01.975162 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:01.975318 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:01.976080 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:01.977052 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:01.977561 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:01.977625 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:01.977686 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:01.977822 (+ 136us) server_negotiation.cc:299] Negotiation successful
0619 20:35:01.977863 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3869,"thread_start_us":82,"threads_started":1}
I20260619 20:35:01.977955 20556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:01.970818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:41803 (local address 127.0.0.1:50392)
0619 20:35:01.974843 (+ 4025us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:01.974877 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:01.974976 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:01.975178 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:01.975180 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:01.975182 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:01.975257 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:01.975259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:01.976155 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:01.976156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:01.976971 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:01.976973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:01.977637 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:01.977638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:01.977727 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:01.977741 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 20:35:01.977802 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 20:35:01.977863 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4001,"thread_start_us":100,"threads_started":1}
I20260619 20:35:01.978758 20482 tablet_service.cc:1514] Processing DeleteTablet for tablet c844af98e96c49d3ab712f971eb83f28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:34:41 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:50392
I20260619 20:35:01.979509 20558 tablet_replica.cc:330] stopping tablet replica
I20260619 20:35:01.979650 20558 raft_consensus.cc:2229] T c844af98e96c49d3ab712f971eb83f28 P cc0e55438ef34044b5879dd1978aa101 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:35:01.979736 20558 raft_consensus.cc:2258] T c844af98e96c49d3ab712f971eb83f28 P cc0e55438ef34044b5879dd1978aa101 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:35:01.980098 20558 ts_tablet_manager.cc:1905] T c844af98e96c49d3ab712f971eb83f28 P cc0e55438ef34044b5879dd1978aa101: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:35:01.982913 20558 ts_tablet_manager.cc:1918] T c844af98e96c49d3ab712f971eb83f28 P cc0e55438ef34044b5879dd1978aa101: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:35:01.983002 20558 log.cc:1196] T c844af98e96c49d3ab712f971eb83f28 P cc0e55438ef34044b5879dd1978aa101: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/wal/wals/c844af98e96c49d3ab712f971eb83f28
I20260619 20:35:01.983225 20558 ts_tablet_manager.cc:1939] T c844af98e96c49d3ab712f971eb83f28 P cc0e55438ef34044b5879dd1978aa101: Deleting consensus metadata
I20260619 20:35:01.983979 20346 catalog_manager.cc:4841] TS cc0e55438ef34044b5879dd1978aa101 (127.4.83.65:41803): tablet c844af98e96c49d3ab712f971eb83f28 (table test-table [id=502cebb10dc24d738c8db5c987269059]) successfully deleted
W20260619 20:35:02.052042 20554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:02.052278 20554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:02.052305 20554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:02.053880 20554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:02.054266 20554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:02.054342 20554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:35:02.057121 20554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:36871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:36025
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=43729
--tserver_master_addrs=127.4.83.126:35383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:02.057615 20554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:02.058130 20554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:02.063748 20563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:02.063750 20566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:02.063990 20564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:02.064193 20554 server_base.cc:1034] running on GCE node
I20260619 20:35:02.064402 20554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:02.064656 20554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:02.065843 20554 hybrid_clock.cc:648] HybridClock initialized: now 1781901302065818 us; error 44 us; skew 500 ppm
Jun 19 20:35:02 dist-test-slave-4m58 krb5kdc[19774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901302, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:35:02.068794 20554 init.cc:377] Logged in from keytab as kudu/127.4.83.66@KRBTEST.COM (short username kudu)
I20260619 20:35:02.069469 20554 webserver.cc:460] Webserver started at http://127.4.83.66:43729/ using document root <none> and password file <none>
I20260619 20:35:02.069705 20554 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:02.069780 20554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:02.071368 20554 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:02.072023 20573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:02.072211 20554 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:02.072301 20554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "690c4aca7db040bf8a41d8eb551dcffc"
format_stamp: "Formatted at 2026-06-19 20:34:41 on dist-test-slave-4m58"
server_key: "9f0f10da8d7af1a9ea11cb4996d9f604"
server_key_iv: "78e553f137320d382552cf0231b10239"
server_key_version: "encryptionkey@0"
I20260619 20:35:02.072670 20554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:02.090251 20554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:02.092759 20554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:02.092909 20554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:02.093147 20554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:35:02.093462 20554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:35:02.093497 20554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:02.093528 20554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:35:02.093545 20554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:02.099442 20554 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:36025
I20260619 20:35:02.099500 20687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:36025 every 8 connection(s)
I20260619 20:35:02.099895 20554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
Jun 19 20:35:02 dist-test-slave-4m58 krb5kdc[19774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901302, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:35:02.107669 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 20554
I20260619 20:35:02.108031 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:41165
--local_ip_for_outbound_sockets=127.4.83.67
--tserver_master_addrs=127.4.83.126:35383
--webserver_port=33341
--webserver_interface=127.4.83.67
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:36871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:35:02.110232 20552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:02.100661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:49449 (local address 127.4.83.126:35383)
0619 20:35:02.100740 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:02.100742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:02.100963 (+ 221us) server_negotiation.cc:407] Connection header received
0619 20:35:02.101486 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:02.101487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:02.101500 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:02.101563 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:02.102526 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.103118 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.104356 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.104607 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.104677 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:02.107164 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:02.107165 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:02.107166 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:35:02.107174 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:02.108680 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:02.109135 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:02.109137 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:02.109137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:02.109198 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:02.109552 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:02.109553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:02.109553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:02.109688 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:02.109746 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:02.110081 (+ 335us) server_negotiation.cc:299] Negotiation successful
0619 20:35:02.110163 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 20:35:02.110496 20690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:02.100650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.66:49449)
0619 20:35:02.100819 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:02.100862 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:02.101366 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:02.101633 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:02.101637 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:02.102230 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:35:02.102420 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:02.102425 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.103219 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.103220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:02.104249 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:02.104253 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.104723 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.104724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:02.104843 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:02.105447 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:35:02.105462 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:35:02.107066 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:35:02.108786 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:35:02.108797 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:35:02.108799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:35:02.109047 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:35:02.109275 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:35:02.109276 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:35:02.109276 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:35:02.109474 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:35:02.109750 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:35:02.109755 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:35:02.109941 (+ 186us) client_negotiation.cc:769] Sending connection context
0619 20:35:02.110055 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 20:35:02.110248 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260619 20:35:02.111021 20688 heartbeater.cc:347] Connected to a master server at 127.4.83.126:35383
I20260619 20:35:02.111127 20688 heartbeater.cc:464] Registering TS with master...
I20260619 20:35:02.111315 20688 heartbeater.cc:511] Master 127.4.83.126:35383 requested a full tablet report, sending...
I20260619 20:35:02.111788 20359 ts_manager.cc:194] Registered new tserver with Master: 690c4aca7db040bf8a41d8eb551dcffc (127.4.83.66:36025)
I20260619 20:35:02.112203 20359 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.66@KRBTEST.COM'} at 127.4.83.66:49449
W20260619 20:35:02.191737 20691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:02.192010 20691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:02.192040 20691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:02.193508 20691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:02.193572 20691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:02.193602 20691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:35:02.195422 20691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:36871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:41165
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=33341
--tserver_master_addrs=127.4.83.126:35383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:02.195720 20691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:02.195978 20691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:02.201364 20696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:02.201376 20699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:02.201385 20697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:02.201485 20691 server_base.cc:1034] running on GCE node
I20260619 20:35:02.201774 20691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:02.202037 20691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:02.203243 20691 hybrid_clock.cc:648] HybridClock initialized: now 1781901302203180 us; error 80 us; skew 500 ppm
Jun 19 20:35:02 dist-test-slave-4m58 krb5kdc[19774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901302, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:35:02.205813 20691 init.cc:377] Logged in from keytab as kudu/127.4.83.67@KRBTEST.COM (short username kudu)
I20260619 20:35:02.206264 20691 webserver.cc:460] Webserver started at http://127.4.83.67:33341/ using document root <none> and password file <none>
I20260619 20:35:02.206458 20691 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:02.206529 20691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:02.207866 20691 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:02.208395 20706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:02.208544 20691 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:02.208626 20691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b"
format_stamp: "Formatted at 2026-06-19 20:34:41 on dist-test-slave-4m58"
server_key: "3c99ecce406b713e497595539c44b8c1"
server_key_iv: "f8316c5232ca68e96c418270702f395c"
server_key_version: "encryptionkey@0"
I20260619 20:35:02.208961 20691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:02.218837 20691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:02.220981 20691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:02.221120 20691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:02.221340 20691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:35:02.221648 20691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:35:02.221680 20691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:02.221709 20691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:35:02.221732 20691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:02.227363 20691 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:41165
I20260619 20:35:02.227403 20820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:41165 every 8 connection(s)
I20260619 20:35:02.227730 20691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
Jun 19 20:35:02 dist-test-slave-4m58 krb5kdc[19774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901302, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:35:02.233717 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 20691
I20260619 20:35:02.237066 20552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:02.228439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:40603 (local address 127.4.83.126:35383)
0619 20:35:02.228513 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:02.228514 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:02.228689 (+ 175us) server_negotiation.cc:407] Connection header received
0619 20:35:02.229161 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:02.229162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:02.229172 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:02.229231 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:02.230096 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.230594 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.231642 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.231975 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.232042 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:02.234213 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:02.234215 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:02.234215 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:35:02.234223 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:02.235702 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:02.236186 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:02.236186 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:02.236187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:02.236250 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:02.236469 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:02.236469 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:02.236470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:02.236597 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:02.236657 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:02.236875 (+ 218us) server_negotiation.cc:299] Negotiation successful
0619 20:35:02.236959 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 20:35:02.237166 20823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:02.228427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.67:40603)
0619 20:35:02.228573 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:02.228613 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:02.229073 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:02.229279 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:02.229282 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:02.229825 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:35:02.230013 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:02.230017 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.230727 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.230728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:02.231547 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:02.231550 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.232068 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.232069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:02.232186 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:02.232658 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:35:02.232667 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:35:02.234117 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:35:02.235785 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:35:02.235793 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:35:02.235795 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:35:02.236112 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:35:02.236311 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:35:02.236311 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:35:02.236312 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:35:02.236413 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:35:02.236655 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:35:02.236658 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:35:02.236770 (+ 112us) client_negotiation.cc:769] Sending connection context
0619 20:35:02.236843 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 20:35:02.236966 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":71,"threads_started":1}
I20260619 20:35:02.237668 20821 heartbeater.cc:347] Connected to a master server at 127.4.83.126:35383
I20260619 20:35:02.237758 20821 heartbeater.cc:464] Registering TS with master...
I20260619 20:35:02.237896 20821 heartbeater.cc:511] Master 127.4.83.126:35383 requested a full tablet report, sending...
I20260619 20:35:02.238268 20359 ts_manager.cc:194] Registered new tserver with Master: e505f8b1bfb04b5f9d4d6e0c54fb4d1b (127.4.83.67:41165)
I20260619 20:35:02.238736 20359 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.67@KRBTEST.COM'} at 127.4.83.67:40603
I20260619 20:35:02.240937 20825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:02.234174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40734 (local address 127.4.83.126:35383)
0619 20:35:02.234274 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:02.234276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:02.234365 (+ 89us) server_negotiation.cc:407] Connection header received
0619 20:35:02.234666 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:02.234667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:02.234678 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:02.234730 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:02.235482 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.236143 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.237097 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.237420 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.237483 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:02.238789 (+ 1306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:02.238791 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:02.238791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:35:02.238798 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:02.240114 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:02.240397 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:02.240397 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:02.240398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:02.240442 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:02.240596 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:02.240597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:02.240597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:02.240683 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:02.240790 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:02.240815 (+ 25us) server_negotiation.cc:299] Negotiation successful
0619 20:35:02.240871 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":56,"threads_started":1}
I20260619 20:35:02.241551 4429 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:35:02.244221 20825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:02.241868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40748 (local address 127.4.83.126:35383)
0619 20:35:02.241907 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:02.241909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:02.241979 (+ 70us) server_negotiation.cc:407] Connection header received
0619 20:35:02.242057 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:02.242058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:02.242084 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:02.242130 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:35:02.242211 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.242663 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.243362 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.243590 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.243660 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:02.243939 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:35:02.243990 (+ 51us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0619 20:35:02.244153 (+ 163us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40748: token expired
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 20:35:02.244290 20825 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40748: token expired
W20260619 20:35:02.244400 20289 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.4.83.126:35383)
I20260619 20:35:02.249977 20825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:02.244680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40764 (local address 127.4.83.126:35383)
0619 20:35:02.244731 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:02.244732 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:02.244740 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:35:02.244758 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:02.244759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:02.244767 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:02.244807 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:02.245245 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.245701 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.246408 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.246597 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.246648 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:02.247728 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:02.247730 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:02.247730 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:35:02.247736 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:02.249031 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:02.249342 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:02.249343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:02.249343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:02.249387 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:02.249516 (+ 129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:02.249516 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:02.249517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:02.249647 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:02.249720 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:02.249856 (+ 136us) server_negotiation.cc:299] Negotiation successful
0619 20:35:02.249915 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 20:35:02.251070 20359 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40764:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:35:02.252656 20359 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:35:02.254710 20482 tablet_service.cc:1467] Processing CreateTablet for tablet 286f6beca789446894ae544ac2dcd4e7 (DEFAULT_TABLE table=test-table [id=29c7ad9418a54f74adb8b6164d9df45f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:35:02.254928 20482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 286f6beca789446894ae544ac2dcd4e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:02.256295 20440 tablet_bootstrap.cc:492] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101: Bootstrap starting.
I20260619 20:35:02.256625 20440 tablet_bootstrap.cc:654] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:02.257201 20440 tablet_bootstrap.cc:492] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101: No bootstrap required, opened a new log
I20260619 20:35:02.257253 20440 ts_tablet_manager.cc:1397] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 20:35:02.257354 20440 raft_consensus.cc:348] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41165 } } peers { permanent_uuid: "690c4aca7db040bf8a41d8eb551dcffc" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 36025 } } peers { permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 } }
I20260619 20:35:02.257432 20440 raft_consensus.cc:374] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:02.257477 20440 raft_consensus.cc:729] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc0e55438ef34044b5879dd1978aa101, State: Initialized, Role: FOLLOWER
I20260619 20:35:02.257514 20440 consensus_queue.cc:260] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41165 } } peers { permanent_uuid: "690c4aca7db040bf8a41d8eb551dcffc" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 36025 } } peers { permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 } }
I20260619 20:35:02.257589 20440 ts_tablet_manager.cc:1428] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 20:35:02.258067 20828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:02.254488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34094 (local address 127.4.83.67:41165)
0619 20:35:02.254676 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:02.254680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:02.254690 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:35:02.254728 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:02.254730 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:02.254746 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:02.254830 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:02.255018 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.255941 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.256972 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.257677 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.257741 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:02.257803 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:02.257933 (+ 130us) server_negotiation.cc:299] Negotiation successful
0619 20:35:02.257977 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":88,"threads_started":1}
I20260619 20:35:02.258816 20556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:02.254385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:41165 (local address 127.0.0.1:34094)
0619 20:35:02.254470 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:02.254482 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:02.254519 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:02.254858 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:02.254860 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:02.254885 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:02.254941 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:02.254943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.256001 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.256002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:02.256887 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:02.256889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.257760 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.257761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:02.257824 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:02.257828 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:35:02.257872 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 20:35:02.258751 (+ 879us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":836}
I20260619 20:35:02.259470 20754 tablet_service.cc:1467] Processing CreateTablet for tablet 286f6beca789446894ae544ac2dcd4e7 (DEFAULT_TABLE table=test-table [id=29c7ad9418a54f74adb8b6164d9df45f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:35:02.259722 20827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:02.254428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:36025 (local address 127.0.0.1:42402)
0619 20:35:02.256335 (+ 1907us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:02.256347 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:02.256402 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:02.256613 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:02.256614 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:02.256616 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:02.256660 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:02.256661 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.257795 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.257796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:02.258682 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:02.258685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.259495 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.259496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:02.259557 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:02.259561 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:35:02.259615 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 20:35:02.259654 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1888,"mutex_wait_us":2,"thread_start_us":92,"threads_started":1}
I20260619 20:35:02.259742 20829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:02.254654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42402 (local address 127.4.83.66:36025)
0619 20:35:02.254799 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:02.254804 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:02.256467 (+ 1663us) server_negotiation.cc:407] Connection header received
0619 20:35:02.256514 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:02.256517 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:02.256533 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:02.256610 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:02.256712 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.257694 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.258931 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.259417 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.259480 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:02.259535 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:02.259630 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 20:35:02.259666 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260619 20:35:02.259819 20754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 286f6beca789446894ae544ac2dcd4e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:02.260277 20621 tablet_service.cc:1467] Processing CreateTablet for tablet 286f6beca789446894ae544ac2dcd4e7 (DEFAULT_TABLE table=test-table [id=29c7ad9418a54f74adb8b6164d9df45f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:35:02.260582 20621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 286f6beca789446894ae544ac2dcd4e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:02.262594 20831 tablet_bootstrap.cc:492] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b: Bootstrap starting.
I20260619 20:35:02.262954 20832 tablet_bootstrap.cc:492] T 286f6beca789446894ae544ac2dcd4e7 P 690c4aca7db040bf8a41d8eb551dcffc: Bootstrap starting.
I20260619 20:35:02.263243 20831 tablet_bootstrap.cc:654] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:02.263619 20832 tablet_bootstrap.cc:654] T 286f6beca789446894ae544ac2dcd4e7 P 690c4aca7db040bf8a41d8eb551dcffc: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:02.263764 20831 log.cc:824] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:02.264253 20832 log.cc:824] T 286f6beca789446894ae544ac2dcd4e7 P 690c4aca7db040bf8a41d8eb551dcffc: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:02.264645 20831 tablet_bootstrap.cc:492] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b: No bootstrap required, opened a new log
I20260619 20:35:02.264707 20831 ts_tablet_manager.cc:1397] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:35:02.264906 20832 tablet_bootstrap.cc:492] T 286f6beca789446894ae544ac2dcd4e7 P 690c4aca7db040bf8a41d8eb551dcffc: No bootstrap required, opened a new log
I20260619 20:35:02.264971 20832 ts_tablet_manager.cc:1397] T 286f6beca789446894ae544ac2dcd4e7 P 690c4aca7db040bf8a41d8eb551dcffc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 20:35:02.266229 20831 raft_consensus.cc:348] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41165 } } peers { permanent_uuid: "690c4aca7db040bf8a41d8eb551dcffc" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 36025 } } peers { permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 } }
I20260619 20:35:02.266278 20832 raft_consensus.cc:348] T 286f6beca789446894ae544ac2dcd4e7 P 690c4aca7db040bf8a41d8eb551dcffc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41165 } } peers { permanent_uuid: "690c4aca7db040bf8a41d8eb551dcffc" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 36025 } } peers { permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 } }
I20260619 20:35:02.266378 20831 raft_consensus.cc:374] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:02.266399 20832 raft_consensus.cc:374] T 286f6beca789446894ae544ac2dcd4e7 P 690c4aca7db040bf8a41d8eb551dcffc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:02.266410 20831 raft_consensus.cc:729] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e505f8b1bfb04b5f9d4d6e0c54fb4d1b, State: Initialized, Role: FOLLOWER
I20260619 20:35:02.266419 20832 raft_consensus.cc:729] T 286f6beca789446894ae544ac2dcd4e7 P 690c4aca7db040bf8a41d8eb551dcffc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 690c4aca7db040bf8a41d8eb551dcffc, State: Initialized, Role: FOLLOWER
I20260619 20:35:02.266527 20831 consensus_queue.cc:260] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41165 } } peers { permanent_uuid: "690c4aca7db040bf8a41d8eb551dcffc" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 36025 } } peers { permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 } }
I20260619 20:35:02.266528 20832 consensus_queue.cc:260] T 286f6beca789446894ae544ac2dcd4e7 P 690c4aca7db040bf8a41d8eb551dcffc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41165 } } peers { permanent_uuid: "690c4aca7db040bf8a41d8eb551dcffc" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 36025 } } peers { permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 } }
I20260619 20:35:02.266808 20831 ts_tablet_manager.cc:1428] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:35:02.266808 20832 ts_tablet_manager.cc:1428] T 286f6beca789446894ae544ac2dcd4e7 P 690c4aca7db040bf8a41d8eb551dcffc: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 20:35:02.266927 20688 heartbeater.cc:503] Master 127.4.83.126:35383 was elected leader, sending a full tablet report...
I20260619 20:35:02.266945 20821 heartbeater.cc:503] Master 127.4.83.126:35383 was elected leader, sending a full tablet report...
W20260619 20:35:02.350445 20689 tablet.cc:2369] T 286f6beca789446894ae544ac2dcd4e7 P 690c4aca7db040bf8a41d8eb551dcffc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:35:02.456540 20550 tablet.cc:2369] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:35:02.478327 20822 tablet.cc:2369] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:35:02.598510 20835 raft_consensus.cc:482] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:35:02.598654 20835 raft_consensus.cc:504] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41165 } } peers { permanent_uuid: "690c4aca7db040bf8a41d8eb551dcffc" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 36025 } } peers { permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 } }
I20260619 20:35:02.599066 20835 leader_election.cc:288] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 690c4aca7db040bf8a41d8eb551dcffc (127.4.83.66:36025), cc0e55438ef34044b5879dd1978aa101 (127.4.83.65:41803)
I20260619 20:35:02.602200 20829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:02.599414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:33715 (local address 127.4.83.66:36025)
0619 20:35:02.599501 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:02.599503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:02.599514 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:35:02.599546 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:02.599547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:02.599560 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:02.599620 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:02.599868 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.600383 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.601452 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.601837 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.601894 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:02.601905 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:02.602093 (+ 188us) server_negotiation.cc:299] Negotiation successful
0619 20:35:02.602134 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 20:35:02.602200 20838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:02.599386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:55071 (local address 127.4.83.65:41803)
0619 20:35:02.599513 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:02.599515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:02.599642 (+ 127us) server_negotiation.cc:407] Connection header received
0619 20:35:02.599673 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:02.599674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:02.599686 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:02.599765 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:02.599993 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.600530 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.601459 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.601850 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.601912 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:02.601924 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:02.602069 (+ 145us) server_negotiation.cc:299] Negotiation successful
0619 20:35:02.602117 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
I20260619 20:35:02.602205 20823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:02.599276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:36025 (local address 127.4.83.67:33715)
0619 20:35:02.599335 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:02.599347 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:02.599418 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:02.599666 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:02.599667 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:02.599670 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:02.599769 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:02.599771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.600459 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.600460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:02.601355 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:02.601358 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.601910 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.601910 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:02.602021 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:02.602024 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:35:02.602077 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 20:35:02.602122 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 20:35:02.602214 20837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:02.599286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:41803 (local address 127.4.83.67:55071)
0619 20:35:02.599525 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:02.599535 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:02.599614 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:02.599802 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:02.599803 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:02.599806 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:02.599909 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:02.599911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.600607 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.600608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:02.601370 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:02.601372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.601931 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.601932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:02.601993 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:02.601997 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:35:02.602047 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 20:35:02.602092 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":60,"thread_start_us":102,"threads_started":1}
I20260619 20:35:02.602573 20642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "286f6beca789446894ae544ac2dcd4e7" candidate_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "690c4aca7db040bf8a41d8eb551dcffc" is_pre_election: true
I20260619 20:35:02.602761 20642 raft_consensus.cc:2454] T 286f6beca789446894ae544ac2dcd4e7 P 690c4aca7db040bf8a41d8eb551dcffc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e505f8b1bfb04b5f9d4d6e0c54fb4d1b in term 0.
I20260619 20:35:02.602782 20503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "286f6beca789446894ae544ac2dcd4e7" candidate_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc0e55438ef34044b5879dd1978aa101" is_pre_election: true
I20260619 20:35:02.602934 20503 raft_consensus.cc:2454] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e505f8b1bfb04b5f9d4d6e0c54fb4d1b in term 0.
I20260619 20:35:02.602990 20709 leader_election.cc:302] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 690c4aca7db040bf8a41d8eb551dcffc, e505f8b1bfb04b5f9d4d6e0c54fb4d1b; no voters:
I20260619 20:35:02.603143 20835 raft_consensus.cc:2781] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:35:02.603200 20835 raft_consensus.cc:482] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:35:02.603223 20835 raft_consensus.cc:3037] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:02.605053 20835 raft_consensus.cc:504] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41165 } } peers { permanent_uuid: "690c4aca7db040bf8a41d8eb551dcffc" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 36025 } } peers { permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 } }
I20260619 20:35:02.605209 20835 leader_election.cc:288] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [CANDIDATE]: Term 1 election: Requested vote from peers 690c4aca7db040bf8a41d8eb551dcffc (127.4.83.66:36025), cc0e55438ef34044b5879dd1978aa101 (127.4.83.65:41803)
I20260619 20:35:02.605417 20503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "286f6beca789446894ae544ac2dcd4e7" candidate_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc0e55438ef34044b5879dd1978aa101"
I20260619 20:35:02.605505 20503 raft_consensus.cc:3037] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:02.605592 20642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "286f6beca789446894ae544ac2dcd4e7" candidate_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "690c4aca7db040bf8a41d8eb551dcffc"
I20260619 20:35:02.605734 20642 raft_consensus.cc:3037] T 286f6beca789446894ae544ac2dcd4e7 P 690c4aca7db040bf8a41d8eb551dcffc [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:02.606217 20503 raft_consensus.cc:2454] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e505f8b1bfb04b5f9d4d6e0c54fb4d1b in term 1.
I20260619 20:35:02.606417 20710 leader_election.cc:302] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc0e55438ef34044b5879dd1978aa101, e505f8b1bfb04b5f9d4d6e0c54fb4d1b; no voters:
I20260619 20:35:02.606472 20642 raft_consensus.cc:2454] T 286f6beca789446894ae544ac2dcd4e7 P 690c4aca7db040bf8a41d8eb551dcffc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e505f8b1bfb04b5f9d4d6e0c54fb4d1b in term 1.
I20260619 20:35:02.606516 20835 raft_consensus.cc:2781] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:35:02.606686 20835 raft_consensus.cc:686] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [term 1 LEADER]: Becoming Leader. State: Replica: e505f8b1bfb04b5f9d4d6e0c54fb4d1b, State: Running, Role: LEADER
I20260619 20:35:02.606814 20835 consensus_queue.cc:237] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41165 } } peers { permanent_uuid: "690c4aca7db040bf8a41d8eb551dcffc" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 36025 } } peers { permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 } }
I20260619 20:35:02.607733 20359 catalog_manager.cc:5495] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b reported cstate change: term changed from 0 to 1, leader changed from <none> to e505f8b1bfb04b5f9d4d6e0c54fb4d1b (127.4.83.67). New cstate: current_term: 1 leader_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "690c4aca7db040bf8a41d8eb551dcffc" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 36025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 } health_report { overall_health: UNKNOWN } } }
I20260619 20:35:02.615646 20828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:02.613165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34098 (local address 127.4.83.67:41165)
0619 20:35:02.613215 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:02.613217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:02.613226 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:35:02.613264 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:02.613265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:02.613283 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:02.613354 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:35:02.613545 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.614062 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.614852 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:02.615058 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:02.615118 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:02.615302 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:35:02.615416 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:35:02.615484 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:02.615554 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 20:35:02.615578 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 20:35:02.617539 20503 raft_consensus.cc:1261] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [term 1 FOLLOWER]: Refusing update from remote peer e505f8b1bfb04b5f9d4d6e0c54fb4d1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:35:02.617539 20642 raft_consensus.cc:1261] T 286f6beca789446894ae544ac2dcd4e7 P 690c4aca7db040bf8a41d8eb551dcffc [term 1 FOLLOWER]: Refusing update from remote peer e505f8b1bfb04b5f9d4d6e0c54fb4d1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:35:02.617928 20835 consensus_queue.cc:1035] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "690c4aca7db040bf8a41d8eb551dcffc" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 36025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:35:02.618088 20839 consensus_queue.cc:1035] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:35:02.620401 20843 mvcc.cc:205] Tried to move back new op lower bound from 7298667735518318592 to 7298667735477932032. Current Snapshot: MvccSnapshot[applied={T|T < 7298667735518318592}]
I20260619 20:35:02.621098 20844 mvcc.cc:205] Tried to move back new op lower bound from 7298667735518318592 to 7298667735477932032. Current Snapshot: MvccSnapshot[applied={T|T < 7298667735518318592}]
I20260619 20:35:02.622568 20358 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:35:02.622663 20358 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:35:02.623413 20358 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad: Sending DeleteTablet for 3 replicas of tablet 286f6beca789446894ae544ac2dcd4e7
I20260619 20:35:02.623720 20482 tablet_service.cc:1514] Processing DeleteTablet for tablet 286f6beca789446894ae544ac2dcd4e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:35:02 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:50392
I20260619 20:35:02.623713 20754 tablet_service.cc:1514] Processing DeleteTablet for tablet 286f6beca789446894ae544ac2dcd4e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:35:02 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:34094
I20260619 20:35:02.623780 20621 tablet_service.cc:1514] Processing DeleteTablet for tablet 286f6beca789446894ae544ac2dcd4e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:35:02 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:42402
I20260619 20:35:02.624070 20854 tablet_replica.cc:330] T 286f6beca789446894ae544ac2dcd4e7 P 690c4aca7db040bf8a41d8eb551dcffc: stopping tablet replica
I20260619 20:35:02.624089 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 20398
I20260619 20:35:02.624172 20854 raft_consensus.cc:2229] T 286f6beca789446894ae544ac2dcd4e7 P 690c4aca7db040bf8a41d8eb551dcffc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:35:02.626770 20854 raft_consensus.cc:2258] T 286f6beca789446894ae544ac2dcd4e7 P 690c4aca7db040bf8a41d8eb551dcffc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:35:02.626976 20852 tablet_replica.cc:330] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b: stopping tablet replica
I20260619 20:35:02.627094 20852 raft_consensus.cc:2229] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:35:02.627199 20852 raft_consensus.cc:2258] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:35:02.627379 20854 ts_tablet_manager.cc:1905] T 286f6beca789446894ae544ac2dcd4e7 P 690c4aca7db040bf8a41d8eb551dcffc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:35:02.629254 20854 ts_tablet_manager.cc:1918] T 286f6beca789446894ae544ac2dcd4e7 P 690c4aca7db040bf8a41d8eb551dcffc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:35:02.629367 20854 log.cc:1196] T 286f6beca789446894ae544ac2dcd4e7 P 690c4aca7db040bf8a41d8eb551dcffc: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/wal/wals/286f6beca789446894ae544ac2dcd4e7
I20260619 20:35:02.629653 20854 ts_tablet_manager.cc:1939] T 286f6beca789446894ae544ac2dcd4e7 P 690c4aca7db040bf8a41d8eb551dcffc: Deleting consensus metadata
I20260619 20:35:02.630155 20345 catalog_manager.cc:4841] TS 690c4aca7db040bf8a41d8eb551dcffc (127.4.83.66:36025): tablet 286f6beca789446894ae544ac2dcd4e7 (table test-table [id=29c7ad9418a54f74adb8b6164d9df45f]) successfully deleted
W20260619 20:35:02.630820 20346 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.83.65:41803 (error 108)
I20260619 20:35:02.631187 20852 ts_tablet_manager.cc:1905] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:35:02.631374 20827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:02.631257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:41803 (local address 127.0.0.1:50394)
0619 20:35:02.631301 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:02.631344 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.65:41803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260619 20:35:02.631537 20346 catalog_manager.cc:4584] TS cc0e55438ef34044b5879dd1978aa101 (127.4.83.65:41803): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 286f6beca789446894ae544ac2dcd4e7: Network error: Client connection negotiation failed: client connection to 127.4.83.65:41803: connect: Connection refused (error 111)
I20260619 20:35:02.632922 20852 ts_tablet_manager.cc:1918] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:35:02.632920 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 20554
I20260619 20:35:02.632989 20852 log.cc:1196] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/wal/wals/286f6beca789446894ae544ac2dcd4e7
I20260619 20:35:02.633654 20852 ts_tablet_manager.cc:1939] T 286f6beca789446894ae544ac2dcd4e7 P e505f8b1bfb04b5f9d4d6e0c54fb4d1b: Deleting consensus metadata
I20260619 20:35:02.634047 20346 catalog_manager.cc:4841] TS e505f8b1bfb04b5f9d4d6e0c54fb4d1b (127.4.83.67:41165): tablet 286f6beca789446894ae544ac2dcd4e7 (table test-table [id=29c7ad9418a54f74adb8b6164d9df45f]) successfully deleted
I20260619 20:35:02.638500 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 20691
I20260619 20:35:02.643013 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 20326
I20260619 20:35:22.647784 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:35383
--webserver_interface=127.4.83.126
--webserver_port=38505
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:36871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:35383
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:35:22.748585 20856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:22.748910 20856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:22.748942 20856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:22.750706 20856 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 20:35:22.750784 20856 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 20:35:22.750797 20856 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 20:35:22.750809 20856 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:35:22.750821 20856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:22.750834 20856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:22.750845 20856 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:35:22.750857 20856 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:35:22.752943 20856 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:36871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:35383
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:35383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=38505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:22.753283 20856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:22.753566 20856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:22.759353 20861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:22.759338 20864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:22.759322 20862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:22.759393 20856 server_base.cc:1034] running on GCE node
I20260619 20:35:22.760003 20856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:22.760392 20856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:22.761547 20856 hybrid_clock.cc:648] HybridClock initialized: now 1781901322761536 us; error 39 us; skew 500 ppm
Jun 19 20:35:22 dist-test-slave-4m58 krb5kdc[19774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901322, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:35:22.768343 20856 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:35:22.768893 20856 webserver.cc:460] Webserver started at http://127.4.83.126:38505/ using document root <none> and password file <none>
I20260619 20:35:22.769147 20856 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:22.769194 20856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:22.770689 20856 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:22.771485 20871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:22.771649 20856 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:22.771732 20856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "3919dc8f8d0a478fa042332a4625c7ad"
format_stamp: "Formatted at 2026-06-19 20:34:40 on dist-test-slave-4m58"
server_key: "8b356c32d6fd560c952378cac8513f31"
server_key_iv: "dba9454d7566d8b55699f26419e09662"
server_key_version: "encryptionkey@0"
I20260619 20:35:22.772197 20856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:22.802925 20856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:22.805263 20856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:22.805413 20856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:22.810600 20856 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:35383
I20260619 20:35:22.810684 20923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:35383 every 8 connection(s)
I20260619 20:35:22.811110 20856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:35:22.812913 20924 sys_catalog.cc:263] Verifying existing consensus state
I20260619 20:35:22.813730 20924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad: Bootstrap starting.
I20260619 20:35:22.815253 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 20856
I20260619 20:35:22.815600 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:41803
--local_ip_for_outbound_sockets=127.4.83.65
--tserver_master_addrs=127.4.83.126:35383
--webserver_port=41743
--webserver_interface=127.4.83.65
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:36871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:35:22.825332 20924 log.cc:824] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:22.828436 20924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:35:22.828832 20924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad: Bootstrap complete.
I20260619 20:35:22.831166 20924 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3919dc8f8d0a478fa042332a4625c7ad" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35383 } }
I20260619 20:35:22.831467 20924 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3919dc8f8d0a478fa042332a4625c7ad, State: Initialized, Role: FOLLOWER
I20260619 20:35:22.831660 20924 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3919dc8f8d0a478fa042332a4625c7ad" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35383 } }
I20260619 20:35:22.831753 20924 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:35:22.831811 20924 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:35:22.831903 20924 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [term 2 FOLLOWER]: Advancing to term 3
I20260619 20:35:22.833681 20924 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3919dc8f8d0a478fa042332a4625c7ad" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35383 } }
I20260619 20:35:22.833818 20924 leader_election.cc:302] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3919dc8f8d0a478fa042332a4625c7ad; no voters:
I20260619 20:35:22.834061 20924 leader_election.cc:288] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 20:35:22.834172 20928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [term 3 FOLLOWER]: Leader election won for term 3
I20260619 20:35:22.834363 20928 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [term 3 LEADER]: Becoming Leader. State: Replica: 3919dc8f8d0a478fa042332a4625c7ad, State: Running, Role: LEADER
I20260619 20:35:22.834407 20924 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:35:22.834486 20928 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3919dc8f8d0a478fa042332a4625c7ad" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35383 } }
I20260619 20:35:22.834800 20929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "3919dc8f8d0a478fa042332a4625c7ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3919dc8f8d0a478fa042332a4625c7ad" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35383 } } }
I20260619 20:35:22.834805 20928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3919dc8f8d0a478fa042332a4625c7ad. Latest consensus state: current_term: 3 leader_uuid: "3919dc8f8d0a478fa042332a4625c7ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3919dc8f8d0a478fa042332a4625c7ad" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35383 } } }
I20260619 20:35:22.835022 20929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:22.835036 20928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:22.835857 20937 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:35:22.837026 20937 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:35:22.837168 20937 catalog_manager.cc:1259] Loaded cluster ID: 1ec1df93e89c42c5a91ce4453f96bc64
I20260619 20:35:22.837205 20937 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:35:22.838431 20937 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:35:22.838671 20937 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad: Loaded TSK: 0
I20260619 20:35:22.838726 20937 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3919dc8f8d0a478fa042332a4625c7ad: Loaded TSK: 1
F20260619 20:35:22.838996 20937 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f0ccab439ad google::LogMessage::Fail() at ??:0
@ 0x7f0ccab4688a google::LogMessage::SendToLog() at ??:0
@ 0x7f0ccab434de google::LogMessage::Flush() at ??:0
@ 0x7f0ccab44a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f0ccd2b0104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f0ccd2add41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f0cd2aa11cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f0cd2aa148e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f0cd2aa1594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f0cd2a67c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f0cd2a71ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f0ccca110d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f0ccca06954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f0ccd86a6db start_thread at ??:0
@ 0x7f0cc964161f clone at ??:0
W20260619 20:35:22.946478 20926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:22.946892 20926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:22.946949 20926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:22.949373 20926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:22.949503 20926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:22.949563 20926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:35:22.952867 20926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:36871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:41803
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=41743
--tserver_master_addrs=127.4.83.126:35383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:22.953322 20926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:22.953665 20926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:22.960601 20966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:22.960587 20965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:22.960739 20969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:22.960626 20926 server_base.cc:1034] running on GCE node
I20260619 20:35:22.961191 20926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:22.961514 20926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:22.962697 20926 hybrid_clock.cc:648] HybridClock initialized: now 1781901322962671 us; error 43 us; skew 500 ppm
Jun 19 20:35:22 dist-test-slave-4m58 krb5kdc[19774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901322, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:35:22.965795 20926 init.cc:377] Logged in from keytab as kudu/127.4.83.65@KRBTEST.COM (short username kudu)
I20260619 20:35:22.966468 20926 webserver.cc:460] Webserver started at http://127.4.83.65:41743/ using document root <none> and password file <none>
I20260619 20:35:22.966758 20926 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:22.966822 20926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:22.970515 20926 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260619 20:35:22.971510 20976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:22.971705 20926 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:22.971822 20926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "cc0e55438ef34044b5879dd1978aa101"
format_stamp: "Formatted at 2026-06-19 20:34:41 on dist-test-slave-4m58"
server_key: "22626140c19773465a6a32386c27666e"
server_key_iv: "25d4a6cf7bde51ca64a76d31cd3e5062"
server_key_version: "encryptionkey@0"
I20260619 20:35:22.972340 20926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:22.983314 20926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:22.985859 20926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:22.986032 20926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:22.986294 20926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:35:22.986812 20983 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 20:35:22.989573 20926 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 20:35:22.989661 20926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260619 20:35:22.989709 20926 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 20:35:22.990571 20926 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 20:35:22.990610 20926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 20:35:22.990727 20983 tablet_bootstrap.cc:492] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101: Bootstrap starting.
I20260619 20:35:22.999317 20926 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:41803
I20260619 20:35:22.999528 21092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:41803 every 8 connection(s)
I20260619 20:35:23.000120 20926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
I20260619 20:35:23.002487 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 20926
I20260619 20:35:23.002841 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:36025
--local_ip_for_outbound_sockets=127.4.83.66
--tserver_master_addrs=127.4.83.126:35383
--webserver_port=43729
--webserver_interface=127.4.83.66
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:36871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:35:23.005359 20983 log.cc:824] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:23.008358 20983 tablet_bootstrap.cc:492] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:35:23.008685 20983 tablet_bootstrap.cc:492] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101: Bootstrap complete.
I20260619 20:35:23.009532 20983 ts_tablet_manager.cc:1397] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.014s
I20260619 20:35:23.010854 20983 raft_consensus.cc:348] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41165 } } peers { permanent_uuid: "690c4aca7db040bf8a41d8eb551dcffc" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 36025 } } peers { permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 } }
I20260619 20:35:23.011157 20983 raft_consensus.cc:729] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc0e55438ef34044b5879dd1978aa101, State: Initialized, Role: FOLLOWER
I20260619 20:35:23.011353 20983 consensus_queue.cc:260] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41165 } } peers { permanent_uuid: "690c4aca7db040bf8a41d8eb551dcffc" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 36025 } } peers { permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 } }
I20260619 20:35:23.011693 20983 ts_tablet_manager.cc:1428] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
W20260619 20:35:23.136224 21095 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:35:23.001226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.65:54903)
0619 20:35:23.001450 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:23.001503 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:23.002270 (+ 767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:23.135962 (+133692us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":135,"threads_started":1}
W20260619 20:35:23.136518 21093 heartbeater.cc:644] Failed to heartbeat to 127.4.83.126:35383 (0 consecutive failures): Network error: Failed to ping master at 127.4.83.126:35383: Client connection negotiation failed: client connection to 127.4.83.126:35383: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 20:35:23.137025 21095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:23.136888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.65:35085)
0619 20:35:23.136949 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:23.136997 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 20:35:23.137377 21095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:23.137294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.65:40607)
0619 20:35:23.137336 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:23.137358 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 20:35:23.137478 21093 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 20:35:23.139118 21096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:23.139405 21096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:23.139448 21096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:23.141661 21096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:23.141772 21096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:23.141822 21096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:35:23.144174 21096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:36871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:36025
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=43729
--tserver_master_addrs=127.4.83.126:35383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:23.144584 21096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:23.144872 21096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:23.151099 21103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:23.151099 21104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:23.151340 21096 server_base.cc:1034] running on GCE node
W20260619 20:35:23.151125 21106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:23.151672 21096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:23.152042 21096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:23.153352 21096 hybrid_clock.cc:648] HybridClock initialized: now 1781901323153329 us; error 40 us; skew 500 ppm
Jun 19 20:35:23 dist-test-slave-4m58 krb5kdc[19774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901323, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:35:23.156947 21096 init.cc:377] Logged in from keytab as kudu/127.4.83.66@KRBTEST.COM (short username kudu)
I20260619 20:35:23.157548 21096 webserver.cc:460] Webserver started at http://127.4.83.66:43729/ using document root <none> and password file <none>
I20260619 20:35:23.157795 21096 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:23.157850 21096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:23.159446 21096 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:35:23.160346 21113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:23.160562 21096 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:23.160670 21096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "690c4aca7db040bf8a41d8eb551dcffc"
format_stamp: "Formatted at 2026-06-19 20:34:41 on dist-test-slave-4m58"
server_key: "9f0f10da8d7af1a9ea11cb4996d9f604"
server_key_iv: "78e553f137320d382552cf0231b10239"
server_key_version: "encryptionkey@0"
I20260619 20:35:23.161135 21096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:23.179970 21096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:23.182926 21096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:23.183135 21096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:23.183425 21096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:35:23.183815 21096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:35:23.183882 21096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:23.183917 21096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:35:23.183935 21096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:23.191425 21096 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:36025
I20260619 20:35:23.191483 21227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:36025 every 8 connection(s)
I20260619 20:35:23.191956 21096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
I20260619 20:35:23.193321 21230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:23.192806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.66:44337)
0619 20:35:23.193042 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:23.193136 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":93,"threads_started":1}
W20260619 20:35:23.193748 21228 heartbeater.cc:644] Failed to heartbeat to 127.4.83.126:35383 (0 consecutive failures): Network error: Failed to ping master at 127.4.83.126:35383: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
I20260619 20:35:23.194039 21230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:23.193951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.66:39355)
0619 20:35:23.193994 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:23.194021 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260619 20:35:23.194283 21230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:23.194212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.66:56875)
0619 20:35:23.194249 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:23.194270 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260619 20:35:23.194380 21228 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 20:35:23.201450 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 21096
I20260619 20:35:23.201812 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:41165
--local_ip_for_outbound_sockets=127.4.83.67
--tserver_master_addrs=127.4.83.126:35383
--webserver_port=33341
--webserver_interface=127.4.83.67
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:36871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:35:23.287662 21231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:23.287947 21231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:23.287974 21231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:23.289229 21231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:23.289283 21231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:23.289311 21231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:35:23.290935 21231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:36871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:41165
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=33341
--tserver_master_addrs=127.4.83.126:35383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:23.291157 21231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:23.291369 21231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:23.296981 21236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:23.297063 21237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:23.297230 21231 server_base.cc:1034] running on GCE node
W20260619 20:35:23.297245 21239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:23.297586 21231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:23.297945 21231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:23.299154 21231 hybrid_clock.cc:648] HybridClock initialized: now 1781901323299143 us; error 35 us; skew 500 ppm
Jun 19 20:35:23 dist-test-slave-4m58 krb5kdc[19774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901323, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:35:23.302706 21231 init.cc:377] Logged in from keytab as kudu/127.4.83.67@KRBTEST.COM (short username kudu)
I20260619 20:35:23.303366 21231 webserver.cc:460] Webserver started at http://127.4.83.67:33341/ using document root <none> and password file <none>
I20260619 20:35:23.303766 21231 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:23.303974 21231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:23.305909 21231 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:23.306850 21246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:23.307093 21231 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:23.307220 21231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b"
format_stamp: "Formatted at 2026-06-19 20:34:41 on dist-test-slave-4m58"
server_key: "3c99ecce406b713e497595539c44b8c1"
server_key_iv: "f8316c5232ca68e96c418270702f395c"
server_key_version: "encryptionkey@0"
I20260619 20:35:23.307765 21231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:23.323551 21231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:23.326076 21231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:23.326236 21231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:23.326507 21231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:35:23.326882 21231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:35:23.326920 21231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:23.326944 21231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:35:23.326956 21231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:23.334678 21231 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:41165
I20260619 20:35:23.334798 21360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:41165 every 8 connection(s)
I20260619 20:35:23.335157 21231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
I20260619 20:35:23.336692 21363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:23.336086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.67:41417)
0619 20:35:23.336315 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:23.336431 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":83,"threads_started":1}
W20260619 20:35:23.337126 21361 heartbeater.cc:644] Failed to heartbeat to 127.4.83.126:35383 (0 consecutive failures): Network error: Failed to ping master at 127.4.83.126:35383: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
I20260619 20:35:23.337567 21363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:23.337426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.67:55735)
0619 20:35:23.337498 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:23.337540 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 20:35:23.337936 21363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:23.337826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.67:58779)
0619 20:35:23.337885 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:23.337913 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260619 20:35:23.338084 21361 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 20:35:23.338403 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 21231
I20260619 20:35:24.138339 21365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:24.138010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.65:42111)
0619 20:35:24.138200 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:24.138268 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":89,"threads_started":1}
I20260619 20:35:24.195034 21366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:24.194716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.66:36165)
0619 20:35:24.194890 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:24.194950 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":74,"threads_started":1}
I20260619 20:35:24.338572 21367 raft_consensus.cc:482] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:35:24.338757 21368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:24.338534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.67:57997)
0619 20:35:24.338640 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:24.338678 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":48,"threads_started":1}
I20260619 20:35:24.338735 21367 raft_consensus.cc:504] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41165 } } peers { permanent_uuid: "690c4aca7db040bf8a41d8eb551dcffc" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 36025 } } peers { permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 } }
I20260619 20:35:24.339164 21367 leader_election.cc:288] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e505f8b1bfb04b5f9d4d6e0c54fb4d1b (127.4.83.67:41165), 690c4aca7db040bf8a41d8eb551dcffc (127.4.83.66:36025)
Jun 19 20:35:24 dist-test-slave-4m58 krb5kdc[19774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901322, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for kudu/127.4.83.67@KRBTEST.COM
Jun 19 20:35:24 dist-test-slave-4m58 krb5kdc[19774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901322, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for kudu/127.4.83.66@KRBTEST.COM
I20260619 20:35:24.351440 21371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:24.339534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:53473 (local address 127.4.83.67:41165)
0619 20:35:24.339778 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:24.339793 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:24.339869 (+ 76us) server_negotiation.cc:407] Connection header received
0619 20:35:24.340571 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:24.340575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:24.340645 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:24.340836 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:24.341814 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:24.342888 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:24.344244 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:24.344713 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:24.344826 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:24.347468 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:24.347475 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:24.347475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:35:24.347492 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:24.349395 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:24.349936 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:24.349939 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:24.349940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:24.350057 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:24.350325 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:24.350325 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:24.350326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:24.350704 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:24.350776 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:24.351161 (+ 385us) server_negotiation.cc:299] Negotiation successful
0619 20:35:24.351322 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":143,"threads_started":1}
I20260619 20:35:24.351454 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:24.339562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:57415 (local address 127.4.83.66:36025)
0619 20:35:24.339777 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:24.339793 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:24.339867 (+ 74us) server_negotiation.cc:407] Connection header received
0619 20:35:24.340583 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:24.340586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:24.340645 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:24.340836 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:24.341853 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:24.343160 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:24.344229 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:24.344673 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:24.344791 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:24.347753 (+ 2962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:24.347759 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:24.347760 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:35:24.347774 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:24.349676 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:24.350097 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:24.350100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:24.350101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:24.350242 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:24.350497 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:24.350498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:24.350498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:24.350886 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:24.350979 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:24.351159 (+ 180us) server_negotiation.cc:299] Negotiation successful
0619 20:35:24.351313 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":59,"threads_started":1}
I20260619 20:35:24.351442 21365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:24.339320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:41165 (local address 127.4.83.65:53473)
0619 20:35:24.339407 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:24.339421 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:24.339570 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:24.340994 (+ 1424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:24.340995 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:24.341393 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:35:24.341674 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:24.341682 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:24.343011 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:24.343012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:24.344143 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:24.344147 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:24.344801 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:24.344802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:24.344899 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:24.345587 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:35:24.345592 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:35:24.347293 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:35:24.349507 (+ 2214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:35:24.349517 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:35:24.349521 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:35:24.349851 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:35:24.350130 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:35:24.350131 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:35:24.350131 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:35:24.350255 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:35:24.350792 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:35:24.350800 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:35:24.350957 (+ 157us) client_negotiation.cc:769] Sending connection context
0619 20:35:24.351079 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 20:35:24.351329 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":23}
I20260619 20:35:24.351562 21369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:24.339411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:36025 (local address 127.4.83.65:57415)
0619 20:35:24.339573 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:24.339586 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:24.339653 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:24.340950 (+ 1297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:24.340953 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:24.341395 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:35:24.341672 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:24.341680 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:24.343292 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:24.343294 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:24.344142 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:24.344145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:24.344763 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:24.344764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:24.344898 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:24.345532 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:35:24.345541 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:35:24.347598 (+ 2057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:35:24.349787 (+ 2189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:35:24.349788 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:35:24.349788 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:35:24.350003 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:35:24.350325 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:35:24.350325 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:35:24.350326 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:35:24.350426 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:35:24.350962 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:35:24.350963 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:35:24.351015 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 20:35:24.351084 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 20:35:24.351421 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":150,"thread_start_us":76,"threads_started":1}
W20260619 20:35:24.352558 20979 leader_election.cc:341] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 690c4aca7db040bf8a41d8eb551dcffc (127.4.83.66:36025): Not found: Tablet not found: 286f6beca789446894ae544ac2dcd4e7
W20260619 20:35:24.352735 20980 leader_election.cc:341] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e505f8b1bfb04b5f9d4d6e0c54fb4d1b (127.4.83.67:41165): Not found: Tablet not found: 286f6beca789446894ae544ac2dcd4e7
I20260619 20:35:24.352790 20980 leader_election.cc:302] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc0e55438ef34044b5879dd1978aa101; no voters: 690c4aca7db040bf8a41d8eb551dcffc, e505f8b1bfb04b5f9d4d6e0c54fb4d1b
I20260619 20:35:24.352903 21367 raft_consensus.cc:2726] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 20:35:25.139437 21372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:25.139144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.65:57409)
0619 20:35:25.139316 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:25.139370 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":107,"threads_started":1}
I20260619 20:35:25.196033 21373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:25.195679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.66:37057)
0619 20:35:25.195909 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:25.195975 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":99,"threads_started":1}
I20260619 20:35:25.339781 21374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:25.339430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.67:42413)
0619 20:35:25.339639 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:25.339699 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":101,"threads_started":1}
I20260619 20:35:26.140352 21375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:26.140050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.65:49361)
0619 20:35:26.140236 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:26.140283 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":80,"threads_started":1}
I20260619 20:35:26.197110 21376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:26.196756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.66:43681)
0619 20:35:26.196974 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:26.197039 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":95,"threads_started":1}
I20260619 20:35:26.340689 21377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:26.340408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.67:42383)
0619 20:35:26.340571 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:26.340628 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":81,"threads_started":1}
I20260619 20:35:26.388160 21378 raft_consensus.cc:482] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:35:26.388298 21378 raft_consensus.cc:504] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41165 } } peers { permanent_uuid: "690c4aca7db040bf8a41d8eb551dcffc" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 36025 } } peers { permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 } }
I20260619 20:35:26.388474 21378 leader_election.cc:288] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e505f8b1bfb04b5f9d4d6e0c54fb4d1b (127.4.83.67:41165), 690c4aca7db040bf8a41d8eb551dcffc (127.4.83.66:36025)
W20260619 20:35:26.389071 20980 leader_election.cc:341] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e505f8b1bfb04b5f9d4d6e0c54fb4d1b (127.4.83.67:41165): Not found: Tablet not found: 286f6beca789446894ae544ac2dcd4e7
W20260619 20:35:26.389182 20979 leader_election.cc:341] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 690c4aca7db040bf8a41d8eb551dcffc (127.4.83.66:36025): Not found: Tablet not found: 286f6beca789446894ae544ac2dcd4e7
I20260619 20:35:26.389242 20979 leader_election.cc:302] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc0e55438ef34044b5879dd1978aa101; no voters: 690c4aca7db040bf8a41d8eb551dcffc, e505f8b1bfb04b5f9d4d6e0c54fb4d1b
I20260619 20:35:26.389317 21378 raft_consensus.cc:2726] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 20:35:27.141306 21379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:27.140989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.65:43903)
0619 20:35:27.141181 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:27.141242 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":84,"threads_started":1}
I20260619 20:35:27.198091 21380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:27.197770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.66:50279)
0619 20:35:27.197962 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:27.198024 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":89,"threads_started":1}
I20260619 20:35:27.341583 21381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:27.341287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.67:56083)
0619 20:35:27.341468 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:27.341512 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":111,"threads_started":1}
I20260619 20:35:28.142360 21382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:28.142015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.65:37071)
0619 20:35:28.142218 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:28.142260 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":102,"threads_started":1}
I20260619 20:35:28.199097 21383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:28.198791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.66:44441)
0619 20:35:28.198971 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:28.199025 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":92,"threads_started":1}
I20260619 20:35:28.342531 21384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:28.342243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.67:37437)
0619 20:35:28.342410 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:28.342479 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":89,"threads_started":1}
I20260619 20:35:28.557426 21385 raft_consensus.cc:482] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:35:28.557566 21385 raft_consensus.cc:504] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41165 } } peers { permanent_uuid: "690c4aca7db040bf8a41d8eb551dcffc" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 36025 } } peers { permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 } }
I20260619 20:35:28.557900 21385 leader_election.cc:288] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e505f8b1bfb04b5f9d4d6e0c54fb4d1b (127.4.83.67:41165), 690c4aca7db040bf8a41d8eb551dcffc (127.4.83.66:36025)
W20260619 20:35:28.558283 20980 leader_election.cc:341] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e505f8b1bfb04b5f9d4d6e0c54fb4d1b (127.4.83.67:41165): Not found: Tablet not found: 286f6beca789446894ae544ac2dcd4e7
W20260619 20:35:28.558373 20979 leader_election.cc:341] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 690c4aca7db040bf8a41d8eb551dcffc (127.4.83.66:36025): Not found: Tablet not found: 286f6beca789446894ae544ac2dcd4e7
I20260619 20:35:28.558415 20979 leader_election.cc:302] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc0e55438ef34044b5879dd1978aa101; no voters: 690c4aca7db040bf8a41d8eb551dcffc, e505f8b1bfb04b5f9d4d6e0c54fb4d1b
I20260619 20:35:28.558528 21385 raft_consensus.cc:2726] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 20:35:29.143342 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:29.143008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.65:39993)
0619 20:35:29.143204 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:29.143274 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":122,"threads_started":1}
I20260619 20:35:29.200126 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:29.199766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.66:41835)
0619 20:35:29.199996 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:29.200069 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":127,"threads_started":1}
I20260619 20:35:29.345242 21388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:29.344920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.67:49403)
0619 20:35:29.345122 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:29.345174 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":105,"threads_started":1}
I20260619 20:35:30.144379 21389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:30.144027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.65:37165)
0619 20:35:30.144227 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:30.144313 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":68,"threads_started":1}
I20260619 20:35:30.201210 21390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:30.200808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.66:43743)
0619 20:35:30.201001 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:30.201142 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":81,"threads_started":1}
I20260619 20:35:30.346208 21391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:30.345942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.67:33977)
0619 20:35:30.346117 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:30.346161 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":93,"threads_started":1}
I20260619 20:35:31.145310 21392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:31.145021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.65:52801)
0619 20:35:31.145185 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:31.145241 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":85,"threads_started":1}
I20260619 20:35:31.202219 21393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:31.201943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.66:36291)
0619 20:35:31.202098 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:31.202154 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
I20260619 20:35:31.347127 21394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:31.346820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.67:33891)
0619 20:35:31.347011 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:31.347064 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":103,"threads_started":1}
I20260619 20:35:32.146363 21395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:32.145970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.65:50335)
0619 20:35:32.146202 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:32.146296 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":97,"threads_started":1}
I20260619 20:35:32.203200 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:32.202839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.66:48889)
0619 20:35:32.203082 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:32.203143 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":131,"threads_started":1}
I20260619 20:35:32.348142 21397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:32.347816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.67:38827)
0619 20:35:32.348027 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:32.348091 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":119,"threads_started":1}
I20260619 20:35:32.854949 21398 raft_consensus.cc:482] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:35:32.855106 21398 raft_consensus.cc:504] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41165 } } peers { permanent_uuid: "690c4aca7db040bf8a41d8eb551dcffc" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 36025 } } peers { permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 } }
I20260619 20:35:32.855343 21398 leader_election.cc:288] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e505f8b1bfb04b5f9d4d6e0c54fb4d1b (127.4.83.67:41165), 690c4aca7db040bf8a41d8eb551dcffc (127.4.83.66:36025)
W20260619 20:35:32.855773 20980 leader_election.cc:341] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e505f8b1bfb04b5f9d4d6e0c54fb4d1b (127.4.83.67:41165): Not found: Tablet not found: 286f6beca789446894ae544ac2dcd4e7
W20260619 20:35:32.855923 20979 leader_election.cc:341] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 690c4aca7db040bf8a41d8eb551dcffc (127.4.83.66:36025): Not found: Tablet not found: 286f6beca789446894ae544ac2dcd4e7
I20260619 20:35:32.855988 20979 leader_election.cc:302] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc0e55438ef34044b5879dd1978aa101; no voters: 690c4aca7db040bf8a41d8eb551dcffc, e505f8b1bfb04b5f9d4d6e0c54fb4d1b
I20260619 20:35:32.856065 21398 raft_consensus.cc:2726] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 20:35:33.147523 21399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:33.147179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.65:45683)
0619 20:35:33.147387 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:33.147444 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":97,"threads_started":1}
I20260619 20:35:33.204249 21400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:33.203936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.66:50381)
0619 20:35:33.204128 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:33.204183 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":87,"threads_started":1}
I20260619 20:35:33.349074 21401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:33.348779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.67:41305)
0619 20:35:33.348958 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:33.349011 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":100,"threads_started":1}
I20260619 20:35:34.148447 21402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:34.148161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.65:44627)
0619 20:35:34.148340 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:34.148395 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":90,"threads_started":1}
I20260619 20:35:34.205076 21403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:34.204803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.66:51949)
0619 20:35:34.204980 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:34.205025 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":71,"threads_started":1}
I20260619 20:35:34.349895 21404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:34.349635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.67:34649)
0619 20:35:34.349796 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:34.349846 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":73,"threads_started":1}
I20260619 20:35:35.149435 21405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:35.149123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.65:40863)
0619 20:35:35.149307 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:35.149367 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":86,"threads_started":1}
I20260619 20:35:35.206095 21406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:35.205782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.66:57929)
0619 20:35:35.205974 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:35.206029 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":113,"threads_started":1}
I20260619 20:35:35.350908 21407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:35.350551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.67:36827)
0619 20:35:35.350782 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:35.350843 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":93,"threads_started":1}
I20260619 20:35:36.150455 21408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:36.150149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.65:51503)
0619 20:35:36.150335 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:36.150389 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":85,"threads_started":1}
I20260619 20:35:36.207103 21409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:36.206795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.66:36739)
0619 20:35:36.206978 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:36.207036 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":90,"threads_started":1}
I20260619 20:35:36.253986 21410 raft_consensus.cc:482] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:35:36.254135 21410 raft_consensus.cc:504] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e505f8b1bfb04b5f9d4d6e0c54fb4d1b" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 41165 } } peers { permanent_uuid: "690c4aca7db040bf8a41d8eb551dcffc" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 36025 } } peers { permanent_uuid: "cc0e55438ef34044b5879dd1978aa101" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41803 } }
I20260619 20:35:36.254354 21410 leader_election.cc:288] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e505f8b1bfb04b5f9d4d6e0c54fb4d1b (127.4.83.67:41165), 690c4aca7db040bf8a41d8eb551dcffc (127.4.83.66:36025)
W20260619 20:35:36.254815 20979 leader_election.cc:341] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 690c4aca7db040bf8a41d8eb551dcffc (127.4.83.66:36025): Not found: Tablet not found: 286f6beca789446894ae544ac2dcd4e7
W20260619 20:35:36.254899 20980 leader_election.cc:341] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e505f8b1bfb04b5f9d4d6e0c54fb4d1b (127.4.83.67:41165): Not found: Tablet not found: 286f6beca789446894ae544ac2dcd4e7
I20260619 20:35:36.254971 20980 leader_election.cc:302] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc0e55438ef34044b5879dd1978aa101; no voters: 690c4aca7db040bf8a41d8eb551dcffc, e505f8b1bfb04b5f9d4d6e0c54fb4d1b
I20260619 20:35:36.255049 21410 raft_consensus.cc:2726] T 286f6beca789446894ae544ac2dcd4e7 P cc0e55438ef34044b5879dd1978aa101 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 20:35:36.351953 21411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:36.351598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.67:59403)
0619 20:35:36.351782 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:36.351875 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":85,"threads_started":1}
I20260619 20:35:37.151537 21412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:37.151183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.65:44547)
0619 20:35:37.151413 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:37.151472 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":106,"threads_started":1}
I20260619 20:35:37.208248 21413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:37.207887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.66:55429)
0619 20:35:37.208091 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:37.208147 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":92,"threads_started":1}
I20260619 20:35:37.352933 21414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:37.352575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.67:53739)
0619 20:35:37.352816 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:37.352871 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":86,"threads_started":1}
I20260619 20:35:38.152647 21415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:38.152298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.65:45357)
0619 20:35:38.152524 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:38.152590 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":103,"threads_started":1}
I20260619 20:35:38.209373 21416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:38.208982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:35383 (local address 127.4.83.66:53697)
0619 20:35:38.209231 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:38.209313 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.126:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":118,"threads_started":1}
I20260619 20:35:38.339543 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 20926
I20260619 20:35:38.345058 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 21096
I20260619 20:35:38.350200 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 21231
2026-06-19T20:35:38Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58560 ms)
[----------] 30 tests from SecurityITest (236345 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:35:38 dist-test-slave-4m58 krb5kdc[21419](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:35:38 dist-test-slave-4m58 krb5kdc[21419](info): set up 2 sockets
Jun 19 20:35:38 dist-test-slave-4m58 krb5kdc[21419](info): commencing operation
krb5kdc: starting...
W20260619 20:35:39.316299 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.946s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:35:39 dist-test-slave-4m58 krb5kdc[21419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:35:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:35:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
I20260619 20:35:39.434466 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:42139
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:44175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:42139
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:35:39.518574 21433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:39.518805 21433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:39.518831 21433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:39.520076 21433 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:35:39.520128 21433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:39.520141 21433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:39.520153 21433 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:35:39.520164 21433 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:35:39.522274 21433 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:44175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:42139
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:42139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:39.522553 21433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:39.522765 21433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:35:39.528419 21433 server_base.cc:1034] running on GCE node
W20260619 20:35:39.528467 21438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:39.528357 21439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:39.528352 21441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:39.528782 21433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:39.529116 21433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:39.530298 21433 hybrid_clock.cc:648] HybridClock initialized: now 1781901339530277 us; error 36 us; skew 500 ppm
Jun 19 20:35:39 dist-test-slave-4m58 krb5kdc[21419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901339, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:35:39.532907 21433 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:35:39.533339 21433 webserver.cc:460] Webserver started at http://127.4.83.126:33691/ using document root <none> and password file <none>
I20260619 20:35:39.533576 21433 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:39.533628 21433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:39.533756 21433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:39.535212 21433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "36237c6b8be74206a603bc1d70e914fa"
format_stamp: "Formatted at 2026-06-19 20:35:39 on dist-test-slave-4m58"
server_key: "5b18534797d9c993908292df553d66d0"
server_key_iv: "bc2304088e4dd06f0dd1bd0d61541513"
server_key_version: "encryptionkey@0"
I20260619 20:35:39.535611 21433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "36237c6b8be74206a603bc1d70e914fa"
format_stamp: "Formatted at 2026-06-19 20:35:39 on dist-test-slave-4m58"
server_key: "5b18534797d9c993908292df553d66d0"
server_key_iv: "bc2304088e4dd06f0dd1bd0d61541513"
server_key_version: "encryptionkey@0"
I20260619 20:35:39.537115 21433 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:35:39.538048 21448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:39.538462 21433 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:39.538580 21433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "36237c6b8be74206a603bc1d70e914fa"
format_stamp: "Formatted at 2026-06-19 20:35:39 on dist-test-slave-4m58"
server_key: "5b18534797d9c993908292df553d66d0"
server_key_iv: "bc2304088e4dd06f0dd1bd0d61541513"
server_key_version: "encryptionkey@0"
I20260619 20:35:39.538661 21433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:39.557049 21433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:39.559250 21433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:39.559369 21433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:39.563082 21433 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:42139
I20260619 20:35:39.563119 21500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:42139 every 8 connection(s)
I20260619 20:35:39.563455 21433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:35:39.564380 21501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:39.566612 21501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36237c6b8be74206a603bc1d70e914fa: Bootstrap starting.
I20260619 20:35:39.567171 21501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36237c6b8be74206a603bc1d70e914fa: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:39.567421 21501 log.cc:824] T 00000000000000000000000000000000 P 36237c6b8be74206a603bc1d70e914fa: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:39.568140 21501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36237c6b8be74206a603bc1d70e914fa: No bootstrap required, opened a new log
I20260619 20:35:39.569689 21501 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36237c6b8be74206a603bc1d70e914fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36237c6b8be74206a603bc1d70e914fa" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 42139 } }
I20260619 20:35:39.569840 21501 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36237c6b8be74206a603bc1d70e914fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:39.569880 21501 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36237c6b8be74206a603bc1d70e914fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36237c6b8be74206a603bc1d70e914fa, State: Initialized, Role: FOLLOWER
I20260619 20:35:39.570039 21501 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36237c6b8be74206a603bc1d70e914fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36237c6b8be74206a603bc1d70e914fa" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 42139 } }
I20260619 20:35:39.570112 21501 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36237c6b8be74206a603bc1d70e914fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:35:39.570135 21501 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36237c6b8be74206a603bc1d70e914fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:35:39.570181 21501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36237c6b8be74206a603bc1d70e914fa [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:39.570155 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 21433
I20260619 20:35:39.570266 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:35:39.570443 4429 external_mini_cluster.cc:1428] Setting key 7132796dbdf3e3b9baa8b8f57f174cfa
I20260619 20:35:39.570824 21501 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36237c6b8be74206a603bc1d70e914fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36237c6b8be74206a603bc1d70e914fa" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 42139 } }
I20260619 20:35:39.570937 21501 leader_election.cc:302] T 00000000000000000000000000000000 P 36237c6b8be74206a603bc1d70e914fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36237c6b8be74206a603bc1d70e914fa; no voters:
I20260619 20:35:39.571147 21501 leader_election.cc:288] T 00000000000000000000000000000000 P 36237c6b8be74206a603bc1d70e914fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:35:39.571182 21506 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36237c6b8be74206a603bc1d70e914fa [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:35:39.571348 21506 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36237c6b8be74206a603bc1d70e914fa [term 1 LEADER]: Becoming Leader. State: Replica: 36237c6b8be74206a603bc1d70e914fa, State: Running, Role: LEADER
I20260619 20:35:39.571410 21501 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36237c6b8be74206a603bc1d70e914fa [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:35:39.572000 21506 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36237c6b8be74206a603bc1d70e914fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36237c6b8be74206a603bc1d70e914fa" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 42139 } }
I20260619 20:35:39.572822 21508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36237c6b8be74206a603bc1d70e914fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36237c6b8be74206a603bc1d70e914fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36237c6b8be74206a603bc1d70e914fa" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 42139 } } }
I20260619 20:35:39.573020 21507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36237c6b8be74206a603bc1d70e914fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36237c6b8be74206a603bc1d70e914fa. Latest consensus state: current_term: 1 leader_uuid: "36237c6b8be74206a603bc1d70e914fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36237c6b8be74206a603bc1d70e914fa" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 42139 } } }
I20260619 20:35:39.573252 21508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36237c6b8be74206a603bc1d70e914fa [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:39.573263 21507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36237c6b8be74206a603bc1d70e914fa [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:39.573446 21521 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:35:39.573908 21521 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:35:39.575258 21521 catalog_manager.cc:1347] Generated new cluster ID: 99164def547645e191417ea33968dce9
I20260619 20:35:39.575299 21521 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:35:39 dist-test-slave-4m58 krb5kdc[21419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:35:39.582674 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:39.570912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47902 (local address 127.4.83.126:42139)
0619 20:35:39.571105 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:39.571114 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:39.571191 (+ 77us) server_negotiation.cc:407] Connection header received
0619 20:35:39.571317 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:39.571324 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:39.571387 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:39.571532 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:39.572246 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:39.573758 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:39.574853 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:39.575433 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:39.575896 (+ 463us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:39.578314 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:39.578318 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:39.578318 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:35:39.578331 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:39.580428 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:39.580910 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:39.580913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:39.580913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:39.581037 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:39.581248 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:39.581249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:39.581249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:39.581599 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:39.582213 (+ 614us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:39.582288 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 20:35:39.582438 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":94,"threads_started":1}
I20260619 20:35:39.585009 21521 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:35:39.585538 21521 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:35:39.593093 21521 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36237c6b8be74206a603bc1d70e914fa: Generated new TSK 0
I20260619 20:35:39.593325 21521 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.83.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
I20260619 20:35:39.622114 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:0
--local_ip_for_outbound_sockets=127.4.83.65
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:42139
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:44175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:35:39.703645 21527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:39.703893 21527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:39.703920 21527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:39.705102 21527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:39.705138 21527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:39.705163 21527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:35:39.706637 21527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:44175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:42139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:39.706835 21527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:39.707027 21527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:39.711984 21532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:39.712131 21527 server_base.cc:1034] running on GCE node
W20260619 20:35:39.711994 21533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:39.711994 21535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:39.712459 21527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:39.712692 21527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:39.713850 21527 hybrid_clock.cc:648] HybridClock initialized: now 1781901339713832 us; error 32 us; skew 500 ppm
Jun 19 20:35:39 dist-test-slave-4m58 krb5kdc[21419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901339, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:35:39.716432 21527 init.cc:377] Logged in from keytab as kudu/127.4.83.65@KRBTEST.COM (short username kudu)
I20260619 20:35:39.716881 21527 webserver.cc:460] Webserver started at http://127.4.83.65:35067/ using document root <none> and password file <none>
I20260619 20:35:39.717104 21527 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:39.717154 21527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:39.717283 21527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:39.718329 21527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-0/data/instance:
uuid: "fae5365bba514731b0bcf57c5425f195"
format_stamp: "Formatted at 2026-06-19 20:35:39 on dist-test-slave-4m58"
server_key: "8b1c7c18b1048e15b6f59dc79b486c6f"
server_key_iv: "926de40c6bd45d57d80d528b8a3589be"
server_key_version: "encryptionkey@0"
I20260619 20:35:39.718662 21527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance:
uuid: "fae5365bba514731b0bcf57c5425f195"
format_stamp: "Formatted at 2026-06-19 20:35:39 on dist-test-slave-4m58"
server_key: "8b1c7c18b1048e15b6f59dc79b486c6f"
server_key_iv: "926de40c6bd45d57d80d528b8a3589be"
server_key_version: "encryptionkey@0"
I20260619 20:35:39.720028 21527 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:35:39.720829 21542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:39.721038 21527 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:39.721128 21527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "fae5365bba514731b0bcf57c5425f195"
format_stamp: "Formatted at 2026-06-19 20:35:39 on dist-test-slave-4m58"
server_key: "8b1c7c18b1048e15b6f59dc79b486c6f"
server_key_iv: "926de40c6bd45d57d80d528b8a3589be"
server_key_version: "encryptionkey@0"
I20260619 20:35:39.721195 21527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:39.732713 21527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:39.734933 21527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:39.735055 21527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:39.735244 21527 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:35:39.735534 21527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:35:39.735569 21527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:39.735597 21527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:35:39.735622 21527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:39.741150 21527 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:43735
I20260619 20:35:39.741212 21656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:43735 every 8 connection(s)
I20260619 20:35:39.741528 21527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
I20260619 20:35:39.747099 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 21527
I20260619 20:35:39.747223 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance
I20260619 20:35:39.747423 4429 external_mini_cluster.cc:1428] Setting key a13656329b2ea43f9cdfb7edb1624645
Jun 19 20:35:39 dist-test-slave-4m58 krb5kdc[21419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901339, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:35:39.750911 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:39.742235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:51469 (local address 127.4.83.126:42139)
0619 20:35:39.742295 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:39.742297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:39.742484 (+ 187us) server_negotiation.cc:407] Connection header received
0619 20:35:39.743045 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:39.743046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:39.743061 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:39.743098 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:39.744075 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:39.744588 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:39.745525 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:39.745719 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:39.745773 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:39.748100 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:39.748101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:39.748102 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:35:39.748109 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:39.749596 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:39.750019 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:39.750020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:39.750021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:39.750084 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:39.750310 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:39.750311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:39.750312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:39.750453 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:39.750515 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:39.750742 (+ 227us) server_negotiation.cc:299] Negotiation successful
0619 20:35:39.750823 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 20:35:39.751032 21659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:39.742230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42139 (local address 127.4.83.65:51469)
0619 20:35:39.742371 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:39.742406 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:39.742944 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:39.743159 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:39.743162 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:39.743726 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:35:39.743985 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:39.743990 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:39.744665 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:39.744666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:39.745451 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:39.745453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:39.745780 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:39.745781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:39.745876 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:39.746371 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:35:39.746380 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:35:39.747998 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:35:39.749696 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:35:39.749704 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:35:39.749705 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:35:39.749943 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:35:39.750144 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:35:39.750145 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:35:39.750145 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:35:39.750247 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:35:39.750508 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:35:39.750512 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:35:39.750632 (+ 120us) client_negotiation.cc:769] Sending connection context
0619 20:35:39.750710 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 20:35:39.750836 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":82,"threads_started":1}
I20260619 20:35:39.751540 21657 heartbeater.cc:347] Connected to a master server at 127.4.83.126:42139
I20260619 20:35:39.751636 21657 heartbeater.cc:464] Registering TS with master...
I20260619 20:35:39.751806 21657 heartbeater.cc:511] Master 127.4.83.126:42139 requested a full tablet report, sending...
I20260619 20:35:39.752391 21465 ts_manager.cc:194] Registered new tserver with Master: fae5365bba514731b0bcf57c5425f195 (127.4.83.65:43735)
I20260619 20:35:39.753016 21465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.65@KRBTEST.COM'} at 127.4.83.65:51469
WARNING: no policy specified for kudu/127.4.83.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
I20260619 20:35:39.770434 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:0
--local_ip_for_outbound_sockets=127.4.83.66
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:42139
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:44175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:35:39.850224 21662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:39.850446 21662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:39.850479 21662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:39.851637 21662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:39.851684 21662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:39.851720 21662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:35:39.853194 21662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:44175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:42139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:39.853436 21662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:39.853646 21662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:39.858637 21667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:39.858631 21670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:39.858631 21668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:39.858659 21662 server_base.cc:1034] running on GCE node
I20260619 20:35:39.858932 21662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:39.859155 21662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:39.860325 21662 hybrid_clock.cc:648] HybridClock initialized: now 1781901339860306 us; error 33 us; skew 500 ppm
Jun 19 20:35:39 dist-test-slave-4m58 krb5kdc[21419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901339, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:35:39.862883 21662 init.cc:377] Logged in from keytab as kudu/127.4.83.66@KRBTEST.COM (short username kudu)
I20260619 20:35:39.863329 21662 webserver.cc:460] Webserver started at http://127.4.83.66:42071/ using document root <none> and password file <none>
I20260619 20:35:39.863551 21662 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:39.863600 21662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:39.863725 21662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:39.864818 21662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-1/data/instance:
uuid: "bbe717f8813e457a82347b28031f636d"
format_stamp: "Formatted at 2026-06-19 20:35:39 on dist-test-slave-4m58"
server_key: "ac133b960bdd57f0f61000e728988735"
server_key_iv: "d6d571d6c3e26e0d7394a8b84dfa655d"
server_key_version: "encryptionkey@0"
I20260619 20:35:39.865157 21662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance:
uuid: "bbe717f8813e457a82347b28031f636d"
format_stamp: "Formatted at 2026-06-19 20:35:39 on dist-test-slave-4m58"
server_key: "ac133b960bdd57f0f61000e728988735"
server_key_iv: "d6d571d6c3e26e0d7394a8b84dfa655d"
server_key_version: "encryptionkey@0"
I20260619 20:35:39.866496 21662 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:35:39.867314 21677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:39.867508 21662 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:39.867602 21662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "bbe717f8813e457a82347b28031f636d"
format_stamp: "Formatted at 2026-06-19 20:35:39 on dist-test-slave-4m58"
server_key: "ac133b960bdd57f0f61000e728988735"
server_key_iv: "d6d571d6c3e26e0d7394a8b84dfa655d"
server_key_version: "encryptionkey@0"
I20260619 20:35:39.867681 21662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:39.884941 21662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:39.887076 21662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:39.887224 21662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:39.887461 21662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:35:39.887763 21662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:35:39.887795 21662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:39.887825 21662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:35:39.887880 21662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:39.893678 21662 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:43655
I20260619 20:35:39.893728 21791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:43655 every 8 connection(s)
I20260619 20:35:39.894040 21662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
I20260619 20:35:39.895484 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 21662
I20260619 20:35:39.895596 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance
I20260619 20:35:39.895814 4429 external_mini_cluster.cc:1428] Setting key 863911bc21f77ddadc3a2acd02b2ad1f
Jun 19 20:35:39 dist-test-slave-4m58 krb5kdc[21419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901339, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.83.67@KRBTEST.COM; defaulting to no policy
I20260619 20:35:39.903945 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:39.894768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:44493 (local address 127.4.83.126:42139)
0619 20:35:39.894824 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:39.894826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:39.895084 (+ 258us) server_negotiation.cc:407] Connection header received
0619 20:35:39.895701 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:39.895702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:39.895713 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:39.895753 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:39.896939 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:39.897555 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:39.898860 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:39.899059 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:39.899119 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:39.901296 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:39.901298 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:39.901298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:35:39.901306 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:39.902628 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:39.903004 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:39.903005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:39.903006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:39.903059 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:39.903276 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:39.903277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:39.903277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:39.903413 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:39.903474 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:39.903739 (+ 265us) server_negotiation.cc:299] Negotiation successful
0619 20:35:39.903844 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 20:35:39.904244 21794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:39.894768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42139 (local address 127.4.83.66:44493)
0619 20:35:39.894963 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:39.895000 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:39.895601 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:39.895859 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:39.895864 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:39.896577 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:35:39.896850 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:39.896858 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:39.897649 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:39.897650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:39.898762 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:39.898766 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:39.899135 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:39.899136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:39.899270 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:39.899788 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:35:39.899797 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:35:39.901199 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:35:39.902708 (+ 1509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:35:39.902716 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:35:39.902718 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:35:39.902939 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:35:39.903112 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:35:39.903113 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:35:39.903113 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:35:39.903222 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:35:39.903472 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:35:39.903480 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:35:39.903623 (+ 143us) client_negotiation.cc:769] Sending connection context
0619 20:35:39.903722 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 20:35:39.903917 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":85,"threads_started":1}
I20260619 20:35:39.904731 21792 heartbeater.cc:347] Connected to a master server at 127.4.83.126:42139
I20260619 20:35:39.904811 21792 heartbeater.cc:464] Registering TS with master...
I20260619 20:35:39.905042 21792 heartbeater.cc:511] Master 127.4.83.126:42139 requested a full tablet report, sending...
I20260619 20:35:39.905416 21465 ts_manager.cc:194] Registered new tserver with Master: bbe717f8813e457a82347b28031f636d (127.4.83.66:43655)
I20260619 20:35:39.905792 21465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.66@KRBTEST.COM'} at 127.4.83.66:44493
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
I20260619 20:35:39.920302 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:0
--local_ip_for_outbound_sockets=127.4.83.67
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:42139
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:44175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:35:40.001864 21797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:40.002213 21797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:40.002265 21797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:40.003634 21797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:40.003695 21797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:40.003726 21797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:35:40.005966 21797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:44175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:42139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:40.006412 21797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:40.008077 21797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:40.014508 21805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:40.014554 21803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:40.014559 21802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:40.014591 21797 server_base.cc:1034] running on GCE node
I20260619 20:35:40.015015 21797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:40.015269 21797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:40.016448 21797 hybrid_clock.cc:648] HybridClock initialized: now 1781901340016435 us; error 31 us; skew 500 ppm
Jun 19 20:35:40 dist-test-slave-4m58 krb5kdc[21419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901340, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:35:40.019084 21797 init.cc:377] Logged in from keytab as kudu/127.4.83.67@KRBTEST.COM (short username kudu)
I20260619 20:35:40.019593 21797 webserver.cc:460] Webserver started at http://127.4.83.67:43151/ using document root <none> and password file <none>
I20260619 20:35:40.019819 21797 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:40.019901 21797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:40.020022 21797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:40.021087 21797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-2/data/instance:
uuid: "b8471de6648b444cb395236354aa85d6"
format_stamp: "Formatted at 2026-06-19 20:35:40 on dist-test-slave-4m58"
server_key: "7c37c9f1ef127ef66112adeb08e9f197"
server_key_iv: "acbdca2872ecdf37f25475929af4750a"
server_key_version: "encryptionkey@0"
I20260619 20:35:40.021440 21797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance:
uuid: "b8471de6648b444cb395236354aa85d6"
format_stamp: "Formatted at 2026-06-19 20:35:40 on dist-test-slave-4m58"
server_key: "7c37c9f1ef127ef66112adeb08e9f197"
server_key_iv: "acbdca2872ecdf37f25475929af4750a"
server_key_version: "encryptionkey@0"
I20260619 20:35:40.022768 21797 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:35:40.023607 21812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:40.023940 21797 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:35:40.024034 21797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "b8471de6648b444cb395236354aa85d6"
format_stamp: "Formatted at 2026-06-19 20:35:40 on dist-test-slave-4m58"
server_key: "7c37c9f1ef127ef66112adeb08e9f197"
server_key_iv: "acbdca2872ecdf37f25475929af4750a"
server_key_version: "encryptionkey@0"
I20260619 20:35:40.024116 21797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:40.032156 21797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:40.034271 21797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:40.034420 21797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:40.034672 21797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:35:40.034973 21797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:35:40.035017 21797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:40.035048 21797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:35:40.035070 21797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:40.041663 21797 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:37329
I20260619 20:35:40.041710 21925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:37329 every 8 connection(s)
I20260619 20:35:40.042093 21797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
I20260619 20:35:40.045284 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 21797
I20260619 20:35:40.045389 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance
I20260619 20:35:40.045578 4429 external_mini_cluster.cc:1428] Setting key 561de3dbc53854dc4b3887c122c3dbbd
Jun 19 20:35:40 dist-test-slave-4m58 krb5kdc[21419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901340, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:35:40.052114 21928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.042748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:42139 (local address 127.4.83.67:59791)
0619 20:35:40.042889 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:40.042926 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:40.043425 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:40.043668 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:40.043672 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:40.044278 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:35:40.044490 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:40.044495 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.045189 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.045190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:40.046182 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:40.046186 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.046584 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.046585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:40.046693 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.047323 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:35:40.047331 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:35:40.048769 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:35:40.050380 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:35:40.050395 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:35:40.050397 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:35:40.050651 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:35:40.050915 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:35:40.050917 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:35:40.050917 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:35:40.051042 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:35:40.051423 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:35:40.051429 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:35:40.051588 (+ 159us) client_negotiation.cc:769] Sending connection context
0619 20:35:40.051690 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 20:35:40.051900 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":71,"threads_started":1}
I20260619 20:35:40.052277 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.042772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:59791 (local address 127.4.83.126:42139)
0619 20:35:40.042815 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:40.042817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:40.043007 (+ 190us) server_negotiation.cc:407] Connection header received
0619 20:35:40.043521 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:40.043522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:40.043535 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:40.043594 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:40.044580 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.045098 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.046250 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.046503 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.046554 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.048909 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:40.048910 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:40.048911 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:35:40.048919 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:40.050285 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:40.050755 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:40.050756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:40.050757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:40.050818 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:40.051137 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:40.051188 (+ 51us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:40.051188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:40.051330 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:40.051398 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:40.051802 (+ 404us) server_negotiation.cc:299] Negotiation successful
0619 20:35:40.052052 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:35:40.052687 21926 heartbeater.cc:347] Connected to a master server at 127.4.83.126:42139
I20260619 20:35:40.052781 21926 heartbeater.cc:464] Registering TS with master...
I20260619 20:35:40.052961 21926 heartbeater.cc:511] Master 127.4.83.126:42139 requested a full tablet report, sending...
I20260619 20:35:40.053351 21465 ts_manager.cc:194] Registered new tserver with Master: b8471de6648b444cb395236354aa85d6 (127.4.83.67:37329)
I20260619 20:35:40.053736 21465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.67@KRBTEST.COM'} at 127.4.83.67:59791
I20260619 20:35:40.057201 4429 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:35:40.064339 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.058494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47912 (local address 127.4.83.126:42139)
0619 20:35:40.058547 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:40.058550 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:40.058638 (+ 88us) server_negotiation.cc:407] Connection header received
0619 20:35:40.058705 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:40.058705 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:40.058716 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:40.058760 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:40.059341 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.059876 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.060627 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.060813 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.060867 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.062012 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:40.062013 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:40.062013 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:35:40.062021 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:40.063237 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:40.063534 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:40.063536 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:40.063536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:40.063607 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:40.063766 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:40.063767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:40.063768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:40.063957 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:40.064047 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:40.064167 (+ 120us) server_negotiation.cc:299] Negotiation successful
0619 20:35:40.064246 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 20:35:40.065878 21465 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47912:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:35:40.067488 21465 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:35:40.073853 21935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.069435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:43735 (local address 127.0.0.1:35092)
0619 20:35:40.070042 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:40.070048 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:40.070082 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:40.070321 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:40.070322 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:40.070324 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:40.070383 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:40.070385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.071759 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.071760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:40.072799 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:40.072802 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.073552 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.073553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:40.073632 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.073646 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 20:35:40.073716 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 20:35:40.073770 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"mutex_wait_us":21,"spinlock_wait_cycles":384,"thread_start_us":75,"threads_started":1}
I20260619 20:35:40.073859 21938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.069704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35092 (local address 127.4.83.65:43735)
0619 20:35:40.069839 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:40.069843 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:40.070101 (+ 258us) server_negotiation.cc:407] Connection header received
0619 20:35:40.070150 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:40.070153 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:40.070169 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:40.070237 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:40.070537 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.071646 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.072888 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.073477 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.073555 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.073620 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:40.073730 (+ 110us) server_negotiation.cc:299] Negotiation successful
0619 20:35:40.073772 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":67,"threads_started":1}
I20260619 20:35:40.074100 21936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.069439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:43655 (local address 127.0.0.1:53242)
0619 20:35:40.070139 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:40.070147 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:40.070184 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:40.070475 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:40.070475 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:40.070477 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:40.070508 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:40.070509 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.071430 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.071431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:40.072869 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:40.072872 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.073858 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.073859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:40.073926 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.073930 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:35:40.073978 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 20:35:40.074025 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"spinlock_wait_cycles":384,"thread_start_us":63,"threads_started":1}
I20260619 20:35:40.074261 21937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.069598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58704 (local address 127.4.83.67:37329)
0619 20:35:40.069788 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:40.069794 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:40.069971 (+ 177us) server_negotiation.cc:407] Connection header received
0619 20:35:40.070049 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:40.070051 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:40.070064 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:40.070130 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:40.070362 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.071661 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.073013 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.073703 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.073769 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.073821 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:40.074130 (+ 309us) server_negotiation.cc:299] Negotiation successful
0619 20:35:40.074179 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":96,"threads_started":1}
I20260619 20:35:40.074266 21934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.069435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:37329 (local address 127.0.0.1:58704)
0619 20:35:40.069871 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:40.069909 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:40.069976 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:40.070206 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:40.070212 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:40.070215 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:40.070286 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:40.070288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.071743 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.071744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:40.072935 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:40.072937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.073864 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.073865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:40.074060 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.074063 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:35:40.074112 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 20:35:40.074209 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":38,"thread_start_us":72,"threads_started":1}
I20260619 20:35:40.074539 21939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.069700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53242 (local address 127.4.83.66:43655)
0619 20:35:40.069878 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:40.069880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:40.070272 (+ 392us) server_negotiation.cc:407] Connection header received
0619 20:35:40.070316 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:40.070318 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:40.070331 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:40.070398 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:40.070574 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.071335 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.072967 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.073703 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.073769 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.073821 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:40.074277 (+ 456us) server_negotiation.cc:299] Negotiation successful
0619 20:35:40.074452 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":83,"threads_started":1}
I20260619 20:35:40.074918 21591 tablet_service.cc:1467] Processing CreateTablet for tablet a36aeadf386a4598bf677de182b6a78e (DEFAULT_TABLE table=test-table [id=1dcc289c7815416b8597a445f181e912]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:35:40.075122 21725 tablet_service.cc:1467] Processing CreateTablet for tablet a36aeadf386a4598bf677de182b6a78e (DEFAULT_TABLE table=test-table [id=1dcc289c7815416b8597a445f181e912]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:35:40.075299 21591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a36aeadf386a4598bf677de182b6a78e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:40.075404 21725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a36aeadf386a4598bf677de182b6a78e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:40.076247 21860 tablet_service.cc:1467] Processing CreateTablet for tablet a36aeadf386a4598bf677de182b6a78e (DEFAULT_TABLE table=test-table [id=1dcc289c7815416b8597a445f181e912]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:35:40.076565 21860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a36aeadf386a4598bf677de182b6a78e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:40.078752 21940 tablet_bootstrap.cc:492] T a36aeadf386a4598bf677de182b6a78e P fae5365bba514731b0bcf57c5425f195: Bootstrap starting.
I20260619 20:35:40.079478 21940 tablet_bootstrap.cc:654] T a36aeadf386a4598bf677de182b6a78e P fae5365bba514731b0bcf57c5425f195: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:40.079597 21941 tablet_bootstrap.cc:492] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6: Bootstrap starting.
I20260619 20:35:40.079887 21942 tablet_bootstrap.cc:492] T a36aeadf386a4598bf677de182b6a78e P bbe717f8813e457a82347b28031f636d: Bootstrap starting.
I20260619 20:35:40.080253 21941 tablet_bootstrap.cc:654] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:40.080322 21940 log.cc:824] T a36aeadf386a4598bf677de182b6a78e P fae5365bba514731b0bcf57c5425f195: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:40.080504 21942 tablet_bootstrap.cc:654] T a36aeadf386a4598bf677de182b6a78e P bbe717f8813e457a82347b28031f636d: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:40.080786 21941 log.cc:824] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:40.081060 21942 log.cc:824] T a36aeadf386a4598bf677de182b6a78e P bbe717f8813e457a82347b28031f636d: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:40.081219 21940 tablet_bootstrap.cc:492] T a36aeadf386a4598bf677de182b6a78e P fae5365bba514731b0bcf57c5425f195: No bootstrap required, opened a new log
I20260619 20:35:40.081323 21940 ts_tablet_manager.cc:1397] T a36aeadf386a4598bf677de182b6a78e P fae5365bba514731b0bcf57c5425f195: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 20:35:40.081508 21941 tablet_bootstrap.cc:492] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6: No bootstrap required, opened a new log
I20260619 20:35:40.081579 21941 ts_tablet_manager.cc:1397] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 20:35:40.081720 21942 tablet_bootstrap.cc:492] T a36aeadf386a4598bf677de182b6a78e P bbe717f8813e457a82347b28031f636d: No bootstrap required, opened a new log
I20260619 20:35:40.081782 21942 ts_tablet_manager.cc:1397] T a36aeadf386a4598bf677de182b6a78e P bbe717f8813e457a82347b28031f636d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:35:40.082911 21940 raft_consensus.cc:348] T a36aeadf386a4598bf677de182b6a78e P fae5365bba514731b0bcf57c5425f195 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8471de6648b444cb395236354aa85d6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37329 } } peers { permanent_uuid: "bbe717f8813e457a82347b28031f636d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 43655 } } peers { permanent_uuid: "fae5365bba514731b0bcf57c5425f195" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 43735 } }
I20260619 20:35:40.083055 21940 raft_consensus.cc:374] T a36aeadf386a4598bf677de182b6a78e P fae5365bba514731b0bcf57c5425f195 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:40.083086 21940 raft_consensus.cc:729] T a36aeadf386a4598bf677de182b6a78e P fae5365bba514731b0bcf57c5425f195 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fae5365bba514731b0bcf57c5425f195, State: Initialized, Role: FOLLOWER
I20260619 20:35:40.083226 21940 consensus_queue.cc:260] T a36aeadf386a4598bf677de182b6a78e P fae5365bba514731b0bcf57c5425f195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8471de6648b444cb395236354aa85d6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37329 } } peers { permanent_uuid: "bbe717f8813e457a82347b28031f636d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 43655 } } peers { permanent_uuid: "fae5365bba514731b0bcf57c5425f195" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 43735 } }
I20260619 20:35:40.083521 21657 heartbeater.cc:503] Master 127.4.83.126:42139 was elected leader, sending a full tablet report...
I20260619 20:35:40.083444 21941 raft_consensus.cc:348] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8471de6648b444cb395236354aa85d6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37329 } } peers { permanent_uuid: "bbe717f8813e457a82347b28031f636d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 43655 } } peers { permanent_uuid: "fae5365bba514731b0bcf57c5425f195" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 43735 } }
I20260619 20:35:40.083626 21941 raft_consensus.cc:374] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:40.083670 21941 raft_consensus.cc:729] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8471de6648b444cb395236354aa85d6, State: Initialized, Role: FOLLOWER
I20260619 20:35:40.083652 21942 raft_consensus.cc:348] T a36aeadf386a4598bf677de182b6a78e P bbe717f8813e457a82347b28031f636d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8471de6648b444cb395236354aa85d6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37329 } } peers { permanent_uuid: "bbe717f8813e457a82347b28031f636d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 43655 } } peers { permanent_uuid: "fae5365bba514731b0bcf57c5425f195" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 43735 } }
I20260619 20:35:40.083797 21942 raft_consensus.cc:374] T a36aeadf386a4598bf677de182b6a78e P bbe717f8813e457a82347b28031f636d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:40.083777 21941 consensus_queue.cc:260] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8471de6648b444cb395236354aa85d6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37329 } } peers { permanent_uuid: "bbe717f8813e457a82347b28031f636d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 43655 } } peers { permanent_uuid: "fae5365bba514731b0bcf57c5425f195" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 43735 } }
I20260619 20:35:40.083869 21942 raft_consensus.cc:729] T a36aeadf386a4598bf677de182b6a78e P bbe717f8813e457a82347b28031f636d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbe717f8813e457a82347b28031f636d, State: Initialized, Role: FOLLOWER
I20260619 20:35:40.083979 21942 consensus_queue.cc:260] T a36aeadf386a4598bf677de182b6a78e P bbe717f8813e457a82347b28031f636d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8471de6648b444cb395236354aa85d6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37329 } } peers { permanent_uuid: "bbe717f8813e457a82347b28031f636d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 43655 } } peers { permanent_uuid: "fae5365bba514731b0bcf57c5425f195" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 43735 } }
I20260619 20:35:40.084059 21941 ts_tablet_manager.cc:1428] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260619 20:35:40.084185 21926 heartbeater.cc:503] Master 127.4.83.126:42139 was elected leader, sending a full tablet report...
I20260619 20:35:40.084246 21942 ts_tablet_manager.cc:1428] T a36aeadf386a4598bf677de182b6a78e P bbe717f8813e457a82347b28031f636d: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260619 20:35:40.083513 21940 ts_tablet_manager.cc:1428] T a36aeadf386a4598bf677de182b6a78e P fae5365bba514731b0bcf57c5425f195: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:35:40.084405 21792 heartbeater.cc:503] Master 127.4.83.126:42139 was elected leader, sending a full tablet report...
I20260619 20:35:40.117172 21947 raft_consensus.cc:482] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:35:40.117312 21947 raft_consensus.cc:504] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8471de6648b444cb395236354aa85d6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37329 } } peers { permanent_uuid: "bbe717f8813e457a82347b28031f636d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 43655 } } peers { permanent_uuid: "fae5365bba514731b0bcf57c5425f195" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 43735 } }
I20260619 20:35:40.117659 21947 leader_election.cc:288] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbe717f8813e457a82347b28031f636d (127.4.83.66:43655), fae5365bba514731b0bcf57c5425f195 (127.4.83.65:43735)
I20260619 20:35:40.120532 21939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.117885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:56089 (local address 127.4.83.66:43655)
0619 20:35:40.117932 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:40.117934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:40.117945 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:35:40.118067 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:40.118068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:40.118082 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:40.118146 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:40.118308 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.118799 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.119780 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.120203 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.120259 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.120269 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:40.120417 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 20:35:40.120457 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:35:40.120551 21938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.117987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:39839 (local address 127.4.83.65:43735)
0619 20:35:40.118032 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:40.118034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:40.118086 (+ 52us) server_negotiation.cc:407] Connection header received
0619 20:35:40.118152 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:40.118153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:40.118165 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:40.118223 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:40.118360 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.118821 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.119783 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.120239 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.120319 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.120331 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:40.120448 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 20:35:40.120484 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 20:35:40.120584 21928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.117815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:43655 (local address 127.4.83.67:56089)
0619 20:35:40.117866 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:40.117879 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:40.117953 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:40.118149 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:40.118150 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:40.118152 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:40.118231 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:40.118233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.118871 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.118872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:40.119688 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:40.119690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.120271 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.120272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:40.120352 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.120355 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:35:40.120407 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 20:35:40.120448 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":3}
I20260619 20:35:40.120599 21949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.117815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:43735 (local address 127.4.83.67:39839)
0619 20:35:40.118021 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:40.118031 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:40.118090 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:40.118229 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:40.118230 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:40.118232 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:40.118297 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:40.118298 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.118896 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.118898 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:40.119695 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:40.119697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.120329 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.120329 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:40.120382 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.120385 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:35:40.120438 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 20:35:40.120473 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":83,"thread_start_us":97,"threads_started":1}
I20260619 20:35:40.120968 21611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a36aeadf386a4598bf677de182b6a78e" candidate_uuid: "b8471de6648b444cb395236354aa85d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae5365bba514731b0bcf57c5425f195" is_pre_election: true
I20260619 20:35:40.120975 21745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a36aeadf386a4598bf677de182b6a78e" candidate_uuid: "b8471de6648b444cb395236354aa85d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe717f8813e457a82347b28031f636d" is_pre_election: true
I20260619 20:35:40.121132 21745 raft_consensus.cc:2454] T a36aeadf386a4598bf677de182b6a78e P bbe717f8813e457a82347b28031f636d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8471de6648b444cb395236354aa85d6 in term 0.
I20260619 20:35:40.121132 21611 raft_consensus.cc:2454] T a36aeadf386a4598bf677de182b6a78e P fae5365bba514731b0bcf57c5425f195 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8471de6648b444cb395236354aa85d6 in term 0.
I20260619 20:35:40.121351 21813 leader_election.cc:302] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8471de6648b444cb395236354aa85d6, bbe717f8813e457a82347b28031f636d; no voters:
I20260619 20:35:40.121529 21947 raft_consensus.cc:2781] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:35:40.121569 21947 raft_consensus.cc:482] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:35:40.121591 21947 raft_consensus.cc:3037] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:40.122264 21947 raft_consensus.cc:504] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8471de6648b444cb395236354aa85d6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37329 } } peers { permanent_uuid: "bbe717f8813e457a82347b28031f636d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 43655 } } peers { permanent_uuid: "fae5365bba514731b0bcf57c5425f195" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 43735 } }
I20260619 20:35:40.122398 21947 leader_election.cc:288] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6 [CANDIDATE]: Term 1 election: Requested vote from peers bbe717f8813e457a82347b28031f636d (127.4.83.66:43655), fae5365bba514731b0bcf57c5425f195 (127.4.83.65:43735)
I20260619 20:35:40.122573 21611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a36aeadf386a4598bf677de182b6a78e" candidate_uuid: "b8471de6648b444cb395236354aa85d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae5365bba514731b0bcf57c5425f195"
I20260619 20:35:40.122573 21745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a36aeadf386a4598bf677de182b6a78e" candidate_uuid: "b8471de6648b444cb395236354aa85d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe717f8813e457a82347b28031f636d"
I20260619 20:35:40.122645 21611 raft_consensus.cc:3037] T a36aeadf386a4598bf677de182b6a78e P fae5365bba514731b0bcf57c5425f195 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:40.122648 21745 raft_consensus.cc:3037] T a36aeadf386a4598bf677de182b6a78e P bbe717f8813e457a82347b28031f636d [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:40.123405 21611 raft_consensus.cc:2454] T a36aeadf386a4598bf677de182b6a78e P fae5365bba514731b0bcf57c5425f195 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8471de6648b444cb395236354aa85d6 in term 1.
I20260619 20:35:40.123447 21745 raft_consensus.cc:2454] T a36aeadf386a4598bf677de182b6a78e P bbe717f8813e457a82347b28031f636d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8471de6648b444cb395236354aa85d6 in term 1.
I20260619 20:35:40.123585 21816 leader_election.cc:302] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8471de6648b444cb395236354aa85d6, fae5365bba514731b0bcf57c5425f195; no voters:
I20260619 20:35:40.123706 21947 raft_consensus.cc:2781] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:35:40.123931 21947 raft_consensus.cc:686] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6 [term 1 LEADER]: Becoming Leader. State: Replica: b8471de6648b444cb395236354aa85d6, State: Running, Role: LEADER
I20260619 20:35:40.124032 21947 consensus_queue.cc:237] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8471de6648b444cb395236354aa85d6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37329 } } peers { permanent_uuid: "bbe717f8813e457a82347b28031f636d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 43655 } } peers { permanent_uuid: "fae5365bba514731b0bcf57c5425f195" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 43735 } }
I20260619 20:35:40.125054 21464 catalog_manager.cc:5495] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8471de6648b444cb395236354aa85d6 (127.4.83.67). New cstate: current_term: 1 leader_uuid: "b8471de6648b444cb395236354aa85d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8471de6648b444cb395236354aa85d6" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 37329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bbe717f8813e457a82347b28031f636d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 43655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fae5365bba514731b0bcf57c5425f195" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 43735 } health_report { overall_health: UNKNOWN } } }
I20260619 20:35:40.131655 21937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.129242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58720 (local address 127.4.83.67:37329)
0619 20:35:40.129302 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:40.129304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:40.129311 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:35:40.129340 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:40.129341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:40.129358 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:40.129421 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:35:40.129576 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.130070 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.130804 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.131037 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.131098 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.131274 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:35:40.131378 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:35:40.131447 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:40.131524 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 20:35:40.131545 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 20:35:40.133395 21611 raft_consensus.cc:1261] T a36aeadf386a4598bf677de182b6a78e P fae5365bba514731b0bcf57c5425f195 [term 1 FOLLOWER]: Refusing update from remote peer b8471de6648b444cb395236354aa85d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:35:40.133397 21745 raft_consensus.cc:1261] T a36aeadf386a4598bf677de182b6a78e P bbe717f8813e457a82347b28031f636d [term 1 FOLLOWER]: Refusing update from remote peer b8471de6648b444cb395236354aa85d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:35:40.133741 21950 consensus_queue.cc:1035] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fae5365bba514731b0bcf57c5425f195" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 43735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:35:40.133874 21947 consensus_queue.cc:1035] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbe717f8813e457a82347b28031f636d" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 43655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:35:40.136368 21955 mvcc.cc:205] Tried to move back new op lower bound from 7298667889184063488 to 7298667889148784640. Current Snapshot: MvccSnapshot[applied={T|T < 7298667889184063488}]
I20260619 20:35:40.136601 21954 mvcc.cc:205] Tried to move back new op lower bound from 7298667889184063488 to 7298667889148784640. Current Snapshot: MvccSnapshot[applied={T|T < 7298667889184063488}]
I20260619 20:35:40.138326 21465 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:35:40.138418 21465 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:35:40.139097 21465 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36237c6b8be74206a603bc1d70e914fa: Sending DeleteTablet for 3 replicas of tablet a36aeadf386a4598bf677de182b6a78e
I20260619 20:35:40.139395 21860 tablet_service.cc:1514] Processing DeleteTablet for tablet a36aeadf386a4598bf677de182b6a78e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:35:40 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:58704
I20260619 20:35:40.139421 21591 tablet_service.cc:1514] Processing DeleteTablet for tablet a36aeadf386a4598bf677de182b6a78e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:35:40 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:35092
I20260619 20:35:40.139535 21725 tablet_service.cc:1514] Processing DeleteTablet for tablet a36aeadf386a4598bf677de182b6a78e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:35:40 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:53242
I20260619 20:35:40.139662 21964 tablet_replica.cc:330] T a36aeadf386a4598bf677de182b6a78e P fae5365bba514731b0bcf57c5425f195: stopping tablet replica
I20260619 20:35:40.139765 21964 raft_consensus.cc:2229] T a36aeadf386a4598bf677de182b6a78e P fae5365bba514731b0bcf57c5425f195 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:35:40.139780 21965 tablet_replica.cc:330] T a36aeadf386a4598bf677de182b6a78e P bbe717f8813e457a82347b28031f636d: stopping tablet replica
I20260619 20:35:40.139892 21964 raft_consensus.cc:2258] T a36aeadf386a4598bf677de182b6a78e P fae5365bba514731b0bcf57c5425f195 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:35:40.139981 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 21527
I20260619 20:35:40.140040 21965 raft_consensus.cc:2229] T a36aeadf386a4598bf677de182b6a78e P bbe717f8813e457a82347b28031f636d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:35:40.140617 21965 raft_consensus.cc:2258] T a36aeadf386a4598bf677de182b6a78e P bbe717f8813e457a82347b28031f636d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:35:40.141057 21965 ts_tablet_manager.cc:1905] T a36aeadf386a4598bf677de182b6a78e P bbe717f8813e457a82347b28031f636d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:35:40.142805 21965 ts_tablet_manager.cc:1918] T a36aeadf386a4598bf677de182b6a78e P bbe717f8813e457a82347b28031f636d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:35:40.142885 21965 log.cc:1196] T a36aeadf386a4598bf677de182b6a78e P bbe717f8813e457a82347b28031f636d: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-1/wal/wals/a36aeadf386a4598bf677de182b6a78e
I20260619 20:35:40.143168 21965 ts_tablet_manager.cc:1939] T a36aeadf386a4598bf677de182b6a78e P bbe717f8813e457a82347b28031f636d: Deleting consensus metadata
I20260619 20:35:40.143584 21449 catalog_manager.cc:4841] TS bbe717f8813e457a82347b28031f636d (127.4.83.66:43655): tablet a36aeadf386a4598bf677de182b6a78e (table test-table [id=1dcc289c7815416b8597a445f181e912]) successfully deleted
W20260619 20:35:40.145574 21452 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.83.65:43735 (error 108)
I20260619 20:35:40.145761 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 21662
I20260619 20:35:40.146548 21963 tablet_replica.cc:330] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6: stopping tablet replica
I20260619 20:35:40.146662 21963 raft_consensus.cc:2229] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:35:40.146776 21963 raft_consensus.cc:2258] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:35:40.147181 21963 ts_tablet_manager.cc:1905] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:35:40.148598 21963 ts_tablet_manager.cc:1918] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:35:40.148663 21963 log.cc:1196] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901102012824-4429-0/minicluster-data/ts-2/wal/wals/a36aeadf386a4598bf677de182b6a78e
I20260619 20:35:40.148870 21963 ts_tablet_manager.cc:1939] T a36aeadf386a4598bf677de182b6a78e P b8471de6648b444cb395236354aa85d6: Deleting consensus metadata
I20260619 20:35:40.149390 21450 catalog_manager.cc:4841] TS b8471de6648b444cb395236354aa85d6 (127.4.83.67:37329): tablet a36aeadf386a4598bf677de182b6a78e (table test-table [id=1dcc289c7815416b8597a445f181e912]) successfully deleted
I20260619 20:35:40.152292 21934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.152173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:43735 (local address 127.0.0.1:35104)
0619 20:35:40.152222 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:40.152261 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.65:43735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 20:35:40.152410 21452 catalog_manager.cc:4584] TS fae5365bba514731b0bcf57c5425f195 (127.4.83.65:43735): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a36aeadf386a4598bf677de182b6a78e: Network error: Client connection negotiation failed: client connection to 127.4.83.65:43735: connect: Connection refused (error 111)
I20260619 20:35:40.152832 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 21797
I20260619 20:35:40.157305 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 21433
2026-06-19T20:35:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1805 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-19T20:35:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:35:40Z Disabled control of system clock
I20260619 20:35:40.183279 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:41685
--webserver_interface=127.4.83.126
--webserver_port=0
--builtin_ntp_servers=127.4.83.84:35631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:41685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 20:35:40.264688 21970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:40.264942 21970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:40.264966 21970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:40.266152 21970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:35:40.266193 21970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:40.266204 21970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:40.266215 21970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:35:40.266227 21970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:35:40.267590 21970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:35631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:41685
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:41685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:40.267810 21970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:40.268123 21970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:40.272927 21976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:40.272935 21978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:40.273068 21970 server_base.cc:1034] running on GCE node
W20260619 20:35:40.273216 21975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:40.273537 21970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:40.273888 21970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:40.275063 21970 hybrid_clock.cc:648] HybridClock initialized: now 1781901340275030 us; error 48 us; skew 500 ppm
I20260619 20:35:40.276525 21970 webserver.cc:460] Webserver started at http://127.4.83.126:45959/ using document root <none> and password file <none>
I20260619 20:35:40.276777 21970 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:40.276827 21970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:40.276966 21970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:40.278055 21970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "57037bd30f8044839b490520717fd89c"
format_stamp: "Formatted at 2026-06-19 20:35:40 on dist-test-slave-4m58"
server_key: "0f62167461081bfb8b4b135846a4596c"
server_key_iv: "9fe6cf1b6fa5be684de10dbccec75614"
server_key_version: "encryptionkey@0"
I20260619 20:35:40.278389 21970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "57037bd30f8044839b490520717fd89c"
format_stamp: "Formatted at 2026-06-19 20:35:40 on dist-test-slave-4m58"
server_key: "0f62167461081bfb8b4b135846a4596c"
server_key_iv: "9fe6cf1b6fa5be684de10dbccec75614"
server_key_version: "encryptionkey@0"
I20260619 20:35:40.279740 21970 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:35:40.280557 21984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:40.280728 21970 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:40.280812 21970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "57037bd30f8044839b490520717fd89c"
format_stamp: "Formatted at 2026-06-19 20:35:40 on dist-test-slave-4m58"
server_key: "0f62167461081bfb8b4b135846a4596c"
server_key_iv: "9fe6cf1b6fa5be684de10dbccec75614"
server_key_version: "encryptionkey@0"
I20260619 20:35:40.280879 21970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:40.294695 21970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:40.295032 21970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:40.295161 21970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:40.299077 21970 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:41685
I20260619 20:35:40.299117 22036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:41685 every 8 connection(s)
I20260619 20:35:40.299441 21970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:35:40.300436 22037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:40.302721 22037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57037bd30f8044839b490520717fd89c: Bootstrap starting.
I20260619 20:35:40.303437 22037 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57037bd30f8044839b490520717fd89c: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:40.303751 22037 log.cc:824] T 00000000000000000000000000000000 P 57037bd30f8044839b490520717fd89c: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:40.304539 22037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57037bd30f8044839b490520717fd89c: No bootstrap required, opened a new log
I20260619 20:35:40.306182 22037 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57037bd30f8044839b490520717fd89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57037bd30f8044839b490520717fd89c" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 41685 } }
I20260619 20:35:40.306336 22037 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57037bd30f8044839b490520717fd89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:40.306358 22037 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57037bd30f8044839b490520717fd89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57037bd30f8044839b490520717fd89c, State: Initialized, Role: FOLLOWER
I20260619 20:35:40.306488 22037 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57037bd30f8044839b490520717fd89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57037bd30f8044839b490520717fd89c" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 41685 } }
I20260619 20:35:40.306571 22037 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57037bd30f8044839b490520717fd89c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:35:40.306594 22037 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57037bd30f8044839b490520717fd89c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:35:40.306622 22037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57037bd30f8044839b490520717fd89c [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:40.307160 22037 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57037bd30f8044839b490520717fd89c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57037bd30f8044839b490520717fd89c" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 41685 } }
I20260619 20:35:40.307250 22037 leader_election.cc:302] T 00000000000000000000000000000000 P 57037bd30f8044839b490520717fd89c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57037bd30f8044839b490520717fd89c; no voters:
I20260619 20:35:40.307394 22037 leader_election.cc:288] T 00000000000000000000000000000000 P 57037bd30f8044839b490520717fd89c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:35:40.307459 22040 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57037bd30f8044839b490520717fd89c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:35:40.307631 22037 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57037bd30f8044839b490520717fd89c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:35:40.307657 22040 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57037bd30f8044839b490520717fd89c [term 1 LEADER]: Becoming Leader. State: Replica: 57037bd30f8044839b490520717fd89c, State: Running, Role: LEADER
I20260619 20:35:40.307752 22040 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57037bd30f8044839b490520717fd89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57037bd30f8044839b490520717fd89c" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 41685 } }
I20260619 20:35:40.307942 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 21970
I20260619 20:35:40.308014 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:35:40.308136 22041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57037bd30f8044839b490520717fd89c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57037bd30f8044839b490520717fd89c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57037bd30f8044839b490520717fd89c" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 41685 } } }
I20260619 20:35:40.308211 4429 external_mini_cluster.cc:1428] Setting key 25483c5e4b2231d1a16139726c8e7346
I20260619 20:35:40.308233 22042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57037bd30f8044839b490520717fd89c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57037bd30f8044839b490520717fd89c. Latest consensus state: current_term: 1 leader_uuid: "57037bd30f8044839b490520717fd89c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57037bd30f8044839b490520717fd89c" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 41685 } } }
I20260619 20:35:40.308293 22042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57037bd30f8044839b490520717fd89c [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:40.308234 22041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57037bd30f8044839b490520717fd89c [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:40.308501 22044 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:35:40.309020 22044 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:35:40.310690 22044 catalog_manager.cc:1347] Generated new cluster ID: 7582354144bf41169abc791e6816e757
I20260619 20:35:40.310745 22044 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:35:40.315073 22051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.308842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35724 (local address 127.4.83.126:41685)
0619 20:35:40.309036 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:40.309045 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:40.309149 (+ 104us) server_negotiation.cc:407] Connection header received
0619 20:35:40.309361 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:40.309369 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:40.309442 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:40.309550 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:40.309981 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.311318 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.312488 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.313140 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.313258 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.314033 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:40.314078 (+ 45us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:40.314079 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:35:40.314317 (+ 238us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:40.314363 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:35:40.314379 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:40.314613 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:40.314693 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 20:35:40.314788 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
I20260619 20:35:40.327958 22044 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:35:40.328753 22044 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:35:40.340950 22044 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57037bd30f8044839b490520717fd89c: Generated new TSK 0
I20260619 20:35:40.341181 22044 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:35:40.344323 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:0
--local_ip_for_outbound_sockets=127.4.83.65
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:41685
--builtin_ntp_servers=127.4.83.84:35631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 20:35:40.427387 22061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:40.427641 22061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:40.427677 22061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:40.429113 22061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:40.429170 22061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:40.429210 22061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:35:40.430740 22061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:35631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:41685
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:40.431021 22061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:40.431334 22061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:40.436326 22066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:40.436436 22067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:40.436327 22069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:40.436702 22061 server_base.cc:1034] running on GCE node
I20260619 20:35:40.436885 22061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:40.437119 22061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:40.438289 22061 hybrid_clock.cc:648] HybridClock initialized: now 1781901340438271 us; error 31 us; skew 500 ppm
I20260619 20:35:40.439780 22061 webserver.cc:460] Webserver started at http://127.4.83.65:39643/ using document root <none> and password file <none>
I20260619 20:35:40.440047 22061 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:40.440106 22061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:40.440237 22061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:40.441318 22061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-0/data/instance:
uuid: "c79ddaa847094ee3bf3a8b219cf6157a"
format_stamp: "Formatted at 2026-06-19 20:35:40 on dist-test-slave-4m58"
server_key: "d066dff5d3ce75b74546712d81ff0ce4"
server_key_iv: "f87641ba8161425efdb87c1913a47dea"
server_key_version: "encryptionkey@0"
I20260619 20:35:40.441649 22061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance:
uuid: "c79ddaa847094ee3bf3a8b219cf6157a"
format_stamp: "Formatted at 2026-06-19 20:35:40 on dist-test-slave-4m58"
server_key: "d066dff5d3ce75b74546712d81ff0ce4"
server_key_iv: "f87641ba8161425efdb87c1913a47dea"
server_key_version: "encryptionkey@0"
I20260619 20:35:40.442930 22061 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:35:40.443744 22075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:40.443964 22061 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:40.444047 22061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "c79ddaa847094ee3bf3a8b219cf6157a"
format_stamp: "Formatted at 2026-06-19 20:35:40 on dist-test-slave-4m58"
server_key: "d066dff5d3ce75b74546712d81ff0ce4"
server_key_iv: "f87641ba8161425efdb87c1913a47dea"
server_key_version: "encryptionkey@0"
I20260619 20:35:40.444120 22061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:40.460143 22061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:40.460460 22061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:40.460590 22061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:40.460804 22061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:35:40.461130 22061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:35:40.461163 22061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:40.461194 22061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:35:40.461216 22061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:40.466768 22061 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:41647
I20260619 20:35:40.466825 22188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:41647 every 8 connection(s)
I20260619 20:35:40.467159 22061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
I20260619 20:35:40.469810 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 22061
I20260619 20:35:40.469915 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance
I20260619 20:35:40.470098 4429 external_mini_cluster.cc:1428] Setting key fa4cf5dff9e45f9d6f6c5b07abd526ce
I20260619 20:35:40.471046 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:0
--local_ip_for_outbound_sockets=127.4.83.66
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:41685
--builtin_ntp_servers=127.4.83.84:35631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 20:35:40.473003 22051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.467917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:48867 (local address 127.4.83.126:41685)
0619 20:35:40.467976 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:40.467978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:40.468196 (+ 218us) server_negotiation.cc:407] Connection header received
0619 20:35:40.468691 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:40.468693 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:40.468706 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:40.468777 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:40.469500 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.470258 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.471552 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.471821 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.471926 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.472763 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:40.472765 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:40.472765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:35:40.472770 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:40.472786 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:35:40.472790 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:40.472830 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:40.472913 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 20:35:40.472944 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 20:35:40.473256 22191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.467874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41685 (local address 127.4.83.65:48867)
0619 20:35:40.468050 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:40.468091 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:40.468576 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:40.468841 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:40.468844 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:40.468852 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:35:40.469387 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:40.469392 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.470372 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.470373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:40.471467 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:40.471470 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.471931 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.471931 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:40.472052 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.472605 (+ 553us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:35:40.472605 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:35:40.472646 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:35:40.472677 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:35:40.472843 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:35:40.472847 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:35:40.472850 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:35:40.472916 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 20:35:40.473028 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
I20260619 20:35:40.473826 22189 heartbeater.cc:347] Connected to a master server at 127.4.83.126:41685
I20260619 20:35:40.473932 22189 heartbeater.cc:464] Registering TS with master...
I20260619 20:35:40.474128 22189 heartbeater.cc:511] Master 127.4.83.126:41685 requested a full tablet report, sending...
I20260619 20:35:40.474790 22001 ts_manager.cc:194] Registered new tserver with Master: c79ddaa847094ee3bf3a8b219cf6157a (127.4.83.65:41647)
I20260619 20:35:40.475411 22001 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.83.65:48867
W20260619 20:35:40.555786 22192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:40.556075 22192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:40.556111 22192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:40.557631 22192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:40.557703 22192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:40.557746 22192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:35:40.559319 22192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:35631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:41685
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:40.559595 22192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:40.559968 22192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:40.565290 22197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:40.565290 22200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:40.565296 22198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:40.565603 22192 server_base.cc:1034] running on GCE node
I20260619 20:35:40.565814 22192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:40.566110 22192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:40.567288 22192 hybrid_clock.cc:648] HybridClock initialized: now 1781901340567279 us; error 32 us; skew 500 ppm
I20260619 20:35:40.569015 22192 webserver.cc:460] Webserver started at http://127.4.83.66:46409/ using document root <none> and password file <none>
I20260619 20:35:40.569458 22192 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:40.569550 22192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:40.569690 22192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:40.570837 22192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-1/data/instance:
uuid: "5e5f5d1cbe58486589fa1d147a80ea98"
format_stamp: "Formatted at 2026-06-19 20:35:40 on dist-test-slave-4m58"
server_key: "e0500fce7f75cbbc0494dec3891cc8f5"
server_key_iv: "6c7b8687a7c41441aa3ad68438d5d5b0"
server_key_version: "encryptionkey@0"
I20260619 20:35:40.571192 22192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance:
uuid: "5e5f5d1cbe58486589fa1d147a80ea98"
format_stamp: "Formatted at 2026-06-19 20:35:40 on dist-test-slave-4m58"
server_key: "e0500fce7f75cbbc0494dec3891cc8f5"
server_key_iv: "6c7b8687a7c41441aa3ad68438d5d5b0"
server_key_version: "encryptionkey@0"
I20260619 20:35:40.572575 22192 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:35:40.573572 22206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:40.573784 22192 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:40.573906 22192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "5e5f5d1cbe58486589fa1d147a80ea98"
format_stamp: "Formatted at 2026-06-19 20:35:40 on dist-test-slave-4m58"
server_key: "e0500fce7f75cbbc0494dec3891cc8f5"
server_key_iv: "6c7b8687a7c41441aa3ad68438d5d5b0"
server_key_version: "encryptionkey@0"
I20260619 20:35:40.573976 22192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:40.581346 22192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:40.581633 22192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:40.581753 22192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:40.581962 22192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:35:40.582262 22192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:35:40.582294 22192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:40.582324 22192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:35:40.582347 22192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:40.588477 22192 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:34919
I20260619 20:35:40.588531 22319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:34919 every 8 connection(s)
I20260619 20:35:40.588864 22192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
I20260619 20:35:40.594431 22051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.589854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:51289 (local address 127.4.83.126:41685)
0619 20:35:40.589930 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:40.589932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:40.590145 (+ 213us) server_negotiation.cc:407] Connection header received
0619 20:35:40.590612 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:40.590613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:40.590623 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:40.590677 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:40.591387 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.591989 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.593014 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.593221 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.593287 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.594165 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:40.594166 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:40.594166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:35:40.594171 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:40.594188 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:35:40.594193 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:40.594240 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:40.594334 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 20:35:40.594366 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 20:35:40.594591 22322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.589805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41685 (local address 127.4.83.66:51289)
0619 20:35:40.589966 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:40.590001 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:40.590499 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:40.590741 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:40.590744 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:40.590752 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:35:40.591281 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:40.591285 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.592093 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.592094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:40.592914 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:40.592917 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.593350 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.593351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:40.593490 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.594024 (+ 534us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:35:40.594024 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:35:40.594054 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:35:40.594078 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:35:40.594245 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:35:40.594247 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:35:40.594250 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:35:40.594297 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 20:35:40.594395 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":68,"threads_started":1}
I20260619 20:35:40.595026 22320 heartbeater.cc:347] Connected to a master server at 127.4.83.126:41685
I20260619 20:35:40.595115 22320 heartbeater.cc:464] Registering TS with master...
I20260619 20:35:40.595265 22320 heartbeater.cc:511] Master 127.4.83.126:41685 requested a full tablet report, sending...
I20260619 20:35:40.595625 22001 ts_manager.cc:194] Registered new tserver with Master: 5e5f5d1cbe58486589fa1d147a80ea98 (127.4.83.66:34919)
I20260619 20:35:40.596036 22001 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.83.66:51289
I20260619 20:35:40.596768 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 22192
I20260619 20:35:40.596868 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance
I20260619 20:35:40.597018 4429 external_mini_cluster.cc:1428] Setting key ca7a25e4555fe1962ebef4e9a336e2df
I20260619 20:35:40.597955 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:0
--local_ip_for_outbound_sockets=127.4.83.67
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:41685
--builtin_ntp_servers=127.4.83.84:35631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 20:35:40.679036 22323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:40.679262 22323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:40.679286 22323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:40.680680 22323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:40.680745 22323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:40.680775 22323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:35:40.682346 22323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:35631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:41685
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:40.682617 22323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:40.682926 22323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:40.688091 22331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:40.688091 22328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:40.688093 22329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:40.688393 22323 server_base.cc:1034] running on GCE node
I20260619 20:35:40.688596 22323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:40.688840 22323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:40.690003 22323 hybrid_clock.cc:648] HybridClock initialized: now 1781901340689972 us; error 43 us; skew 500 ppm
I20260619 20:35:40.691342 22323 webserver.cc:460] Webserver started at http://127.4.83.67:38607/ using document root <none> and password file <none>
I20260619 20:35:40.691586 22323 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:40.691637 22323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:40.691772 22323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:40.692919 22323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-2/data/instance:
uuid: "9582eea982e04458897fd5bfaca4db9e"
format_stamp: "Formatted at 2026-06-19 20:35:40 on dist-test-slave-4m58"
server_key: "30edea38f22b35319b1b49b1da3ae185"
server_key_iv: "41c69247db16fb78dd69b64b93076968"
server_key_version: "encryptionkey@0"
I20260619 20:35:40.693289 22323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance:
uuid: "9582eea982e04458897fd5bfaca4db9e"
format_stamp: "Formatted at 2026-06-19 20:35:40 on dist-test-slave-4m58"
server_key: "30edea38f22b35319b1b49b1da3ae185"
server_key_iv: "41c69247db16fb78dd69b64b93076968"
server_key_version: "encryptionkey@0"
I20260619 20:35:40.694640 22323 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:35:40.695534 22337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:40.695724 22323 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:40.695813 22323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "9582eea982e04458897fd5bfaca4db9e"
format_stamp: "Formatted at 2026-06-19 20:35:40 on dist-test-slave-4m58"
server_key: "30edea38f22b35319b1b49b1da3ae185"
server_key_iv: "41c69247db16fb78dd69b64b93076968"
server_key_version: "encryptionkey@0"
I20260619 20:35:40.695921 22323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:40.704779 22323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:40.705096 22323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:40.705221 22323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:40.705431 22323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:35:40.705747 22323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:35:40.705780 22323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:40.705808 22323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:35:40.705832 22323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:40.711387 22323 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:39593
I20260619 20:35:40.711453 22450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:39593 every 8 connection(s)
I20260619 20:35:40.711756 22323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
I20260619 20:35:40.712682 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 22323
I20260619 20:35:40.712791 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance
I20260619 20:35:40.712970 4429 external_mini_cluster.cc:1428] Setting key 1ac7c012d8011f1bb131639bf010cbaf
I20260619 20:35:40.717196 22051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.712574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:41783 (local address 127.4.83.126:41685)
0619 20:35:40.712628 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:40.712630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:40.712850 (+ 220us) server_negotiation.cc:407] Connection header received
0619 20:35:40.713462 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:40.713463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:40.713473 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:40.713517 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:40.714204 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.714704 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.715696 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.715948 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.716006 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.716844 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:40.716845 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:40.716846 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:35:40.716851 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:40.716869 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:35:40.716874 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:40.716961 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:40.717046 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 20:35:40.717081 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 20:35:40.717401 22453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.712518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:41685 (local address 127.4.83.67:41783)
0619 20:35:40.712728 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:40.712771 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:40.713375 (+ 604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:40.713589 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:40.713593 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:40.713603 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:35:40.714111 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:40.714122 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.714790 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.714791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:40.715613 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:40.715616 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.716020 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.716021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:40.716162 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.716687 (+ 525us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:35:40.716687 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:35:40.716721 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:35:40.716746 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:35:40.716969 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:35:40.716975 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:35:40.716979 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:35:40.717044 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 20:35:40.717169 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":93,"threads_started":1}
I20260619 20:35:40.717880 22451 heartbeater.cc:347] Connected to a master server at 127.4.83.126:41685
I20260619 20:35:40.717978 22451 heartbeater.cc:464] Registering TS with master...
I20260619 20:35:40.718159 22451 heartbeater.cc:511] Master 127.4.83.126:41685 requested a full tablet report, sending...
I20260619 20:35:40.718614 22001 ts_manager.cc:194] Registered new tserver with Master: 9582eea982e04458897fd5bfaca4db9e (127.4.83.67:39593)
I20260619 20:35:40.719019 22001 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.83.67:41783
I20260619 20:35:40.724720 4429 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:35:40.729176 22051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.726190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35728 (local address 127.4.83.126:41685)
0619 20:35:40.726284 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:40.726286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:40.726388 (+ 102us) server_negotiation.cc:407] Connection header received
0619 20:35:40.726457 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:40.726458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:40.726472 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:40.726521 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:40.726662 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.727165 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.727961 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.728151 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.728206 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.728909 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:40.728911 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:40.728911 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:35:40.728917 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:40.728936 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:35:40.728941 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:40.728985 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:40.729041 (+ 56us) server_negotiation.cc:299] Negotiation successful
0619 20:35:40.729073 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 20:35:40.730978 22001 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35728:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:35:40.732684 22001 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:35:40.739730 22461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.734901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40274 (local address 127.4.83.65:41647)
0619 20:35:40.735200 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:40.735203 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:40.735338 (+ 135us) server_negotiation.cc:407] Connection header received
0619 20:35:40.735382 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:40.735384 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:40.735396 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:40.735471 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:40.735640 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.736509 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.737696 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.738481 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.738556 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.738648 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:40.739605 (+ 957us) server_negotiation.cc:299] Negotiation successful
0619 20:35:40.739652 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":109,"threads_started":1}
I20260619 20:35:40.739722 22460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.734778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:41647 (local address 127.0.0.1:40274)
0619 20:35:40.735261 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:40.735270 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:40.735324 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:40.735539 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:40.735540 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:40.735541 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:40.735569 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:40.735570 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.736612 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.736613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:40.737581 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:40.737583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.739477 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.739478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:40.739533 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.739537 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:35:40.739583 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 20:35:40.739631 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"mutex_wait_us":2,"thread_start_us":76,"threads_started":1}
I20260619 20:35:40.739723 22459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.734778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:34919 (local address 127.0.0.1:36478)
0619 20:35:40.735117 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:40.735161 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:40.735238 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:40.735473 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:40.735476 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:40.735479 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:40.735559 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:40.735562 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.737378 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.737379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:40.738475 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:40.738478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.739373 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.739374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:40.739455 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.739466 (+ 11us) client_negotiation.cc:769] Sending connection context
0619 20:35:40.739554 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 20:35:40.739615 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":1,"thread_start_us":63,"threads_started":1}
I20260619 20:35:40.740152 22462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.734953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36478 (local address 127.4.83.66:34919)
0619 20:35:40.735123 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:40.735128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:40.735314 (+ 186us) server_negotiation.cc:407] Connection header received
0619 20:35:40.735360 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:40.735362 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:40.735379 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:40.735457 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:40.736603 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.737277 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.738565 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.739275 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.739352 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.739441 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:40.740017 (+ 576us) server_negotiation.cc:299] Negotiation successful
0619 20:35:40.740065 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":71,"threads_started":1}
I20260619 20:35:40.740700 22123 tablet_service.cc:1467] Processing CreateTablet for tablet 7efc5620ae2e4f6baa16b5af96009a36 (DEFAULT_TABLE table=test-table [id=0b946861995f4deabdf55a4e913e6004]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:35:40.740967 22463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.734977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:39593 (local address 127.0.0.1:41348)
0619 20:35:40.735399 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:40.735407 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:40.735447 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:40.737366 (+ 1919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:40.737367 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:40.737369 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:40.737413 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:40.737414 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.738392 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.738393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:40.739482 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:40.739484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.740730 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.740731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:40.740795 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.740799 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:35:40.740853 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 20:35:40.740896 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"mutex_wait_us":3,"thread_start_us":61,"threads_started":1}
I20260619 20:35:40.741086 22123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7efc5620ae2e4f6baa16b5af96009a36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:40.741392 22254 tablet_service.cc:1467] Processing CreateTablet for tablet 7efc5620ae2e4f6baa16b5af96009a36 (DEFAULT_TABLE table=test-table [id=0b946861995f4deabdf55a4e913e6004]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:35:40.741701 22254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7efc5620ae2e4f6baa16b5af96009a36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:40.744119 22464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.735114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41348 (local address 127.4.83.67:39593)
0619 20:35:40.735256 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:40.735261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:40.735501 (+ 240us) server_negotiation.cc:407] Connection header received
0619 20:35:40.735544 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:40.735546 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:40.735560 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:40.735632 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:40.737499 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.738317 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.739577 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.740651 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.740719 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.740804 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:40.743997 (+ 3193us) server_negotiation.cc:299] Negotiation successful
0619 20:35:40.744048 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":64,"threads_started":1}
I20260619 20:35:40.744731 22465 tablet_bootstrap.cc:492] T 7efc5620ae2e4f6baa16b5af96009a36 P c79ddaa847094ee3bf3a8b219cf6157a: Bootstrap starting.
I20260619 20:35:40.744712 22385 tablet_service.cc:1467] Processing CreateTablet for tablet 7efc5620ae2e4f6baa16b5af96009a36 (DEFAULT_TABLE table=test-table [id=0b946861995f4deabdf55a4e913e6004]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:35:40.745060 22385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7efc5620ae2e4f6baa16b5af96009a36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:40.745237 22466 tablet_bootstrap.cc:492] T 7efc5620ae2e4f6baa16b5af96009a36 P 5e5f5d1cbe58486589fa1d147a80ea98: Bootstrap starting.
I20260619 20:35:40.745467 22465 tablet_bootstrap.cc:654] T 7efc5620ae2e4f6baa16b5af96009a36 P c79ddaa847094ee3bf3a8b219cf6157a: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:40.745755 22466 tablet_bootstrap.cc:654] T 7efc5620ae2e4f6baa16b5af96009a36 P 5e5f5d1cbe58486589fa1d147a80ea98: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:40.745775 22465 log.cc:824] T 7efc5620ae2e4f6baa16b5af96009a36 P c79ddaa847094ee3bf3a8b219cf6157a: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:40.746042 22466 log.cc:824] T 7efc5620ae2e4f6baa16b5af96009a36 P 5e5f5d1cbe58486589fa1d147a80ea98: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:40.746582 22465 tablet_bootstrap.cc:492] T 7efc5620ae2e4f6baa16b5af96009a36 P c79ddaa847094ee3bf3a8b219cf6157a: No bootstrap required, opened a new log
I20260619 20:35:40.746685 22465 ts_tablet_manager.cc:1397] T 7efc5620ae2e4f6baa16b5af96009a36 P c79ddaa847094ee3bf3a8b219cf6157a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:35:40.746855 22466 tablet_bootstrap.cc:492] T 7efc5620ae2e4f6baa16b5af96009a36 P 5e5f5d1cbe58486589fa1d147a80ea98: No bootstrap required, opened a new log
I20260619 20:35:40.746927 22466 ts_tablet_manager.cc:1397] T 7efc5620ae2e4f6baa16b5af96009a36 P 5e5f5d1cbe58486589fa1d147a80ea98: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:35:40.748492 22469 tablet_bootstrap.cc:492] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e: Bootstrap starting.
I20260619 20:35:40.749152 22469 tablet_bootstrap.cc:654] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:40.749022 22466 raft_consensus.cc:348] T 7efc5620ae2e4f6baa16b5af96009a36 P 5e5f5d1cbe58486589fa1d147a80ea98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5f5d1cbe58486589fa1d147a80ea98" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34919 } } peers { permanent_uuid: "9582eea982e04458897fd5bfaca4db9e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 39593 } } peers { permanent_uuid: "c79ddaa847094ee3bf3a8b219cf6157a" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41647 } }
I20260619 20:35:40.749198 22466 raft_consensus.cc:374] T 7efc5620ae2e4f6baa16b5af96009a36 P 5e5f5d1cbe58486589fa1d147a80ea98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:40.749233 22466 raft_consensus.cc:729] T 7efc5620ae2e4f6baa16b5af96009a36 P 5e5f5d1cbe58486589fa1d147a80ea98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5f5d1cbe58486589fa1d147a80ea98, State: Initialized, Role: FOLLOWER
I20260619 20:35:40.749365 22466 consensus_queue.cc:260] T 7efc5620ae2e4f6baa16b5af96009a36 P 5e5f5d1cbe58486589fa1d147a80ea98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5f5d1cbe58486589fa1d147a80ea98" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34919 } } peers { permanent_uuid: "9582eea982e04458897fd5bfaca4db9e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 39593 } } peers { permanent_uuid: "c79ddaa847094ee3bf3a8b219cf6157a" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41647 } }
I20260619 20:35:40.749298 22465 raft_consensus.cc:348] T 7efc5620ae2e4f6baa16b5af96009a36 P c79ddaa847094ee3bf3a8b219cf6157a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5f5d1cbe58486589fa1d147a80ea98" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34919 } } peers { permanent_uuid: "9582eea982e04458897fd5bfaca4db9e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 39593 } } peers { permanent_uuid: "c79ddaa847094ee3bf3a8b219cf6157a" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41647 } }
I20260619 20:35:40.749485 22469 log.cc:824] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:40.749504 22465 raft_consensus.cc:374] T 7efc5620ae2e4f6baa16b5af96009a36 P c79ddaa847094ee3bf3a8b219cf6157a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:40.749537 22465 raft_consensus.cc:729] T 7efc5620ae2e4f6baa16b5af96009a36 P c79ddaa847094ee3bf3a8b219cf6157a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c79ddaa847094ee3bf3a8b219cf6157a, State: Initialized, Role: FOLLOWER
I20260619 20:35:40.749640 22466 ts_tablet_manager.cc:1428] T 7efc5620ae2e4f6baa16b5af96009a36 P 5e5f5d1cbe58486589fa1d147a80ea98: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:35:40.749639 22465 consensus_queue.cc:260] T 7efc5620ae2e4f6baa16b5af96009a36 P c79ddaa847094ee3bf3a8b219cf6157a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5f5d1cbe58486589fa1d147a80ea98" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34919 } } peers { permanent_uuid: "9582eea982e04458897fd5bfaca4db9e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 39593 } } peers { permanent_uuid: "c79ddaa847094ee3bf3a8b219cf6157a" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41647 } }
I20260619 20:35:40.749791 22320 heartbeater.cc:503] Master 127.4.83.126:41685 was elected leader, sending a full tablet report...
I20260619 20:35:40.749873 22465 ts_tablet_manager.cc:1428] T 7efc5620ae2e4f6baa16b5af96009a36 P c79ddaa847094ee3bf3a8b219cf6157a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:35:40.749899 22189 heartbeater.cc:503] Master 127.4.83.126:41685 was elected leader, sending a full tablet report...
I20260619 20:35:40.750571 22469 tablet_bootstrap.cc:492] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e: No bootstrap required, opened a new log
I20260619 20:35:40.750636 22469 ts_tablet_manager.cc:1397] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:35:40.752733 22469 raft_consensus.cc:348] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5f5d1cbe58486589fa1d147a80ea98" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34919 } } peers { permanent_uuid: "9582eea982e04458897fd5bfaca4db9e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 39593 } } peers { permanent_uuid: "c79ddaa847094ee3bf3a8b219cf6157a" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41647 } }
I20260619 20:35:40.752875 22469 raft_consensus.cc:374] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:40.752907 22469 raft_consensus.cc:729] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9582eea982e04458897fd5bfaca4db9e, State: Initialized, Role: FOLLOWER
I20260619 20:35:40.753014 22469 consensus_queue.cc:260] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5f5d1cbe58486589fa1d147a80ea98" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34919 } } peers { permanent_uuid: "9582eea982e04458897fd5bfaca4db9e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 39593 } } peers { permanent_uuid: "c79ddaa847094ee3bf3a8b219cf6157a" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41647 } }
I20260619 20:35:40.753255 22469 ts_tablet_manager.cc:1428] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:35:40.753306 22451 heartbeater.cc:503] Master 127.4.83.126:41685 was elected leader, sending a full tablet report...
W20260619 20:35:40.839622 22321 tablet.cc:2369] T 7efc5620ae2e4f6baa16b5af96009a36 P 5e5f5d1cbe58486589fa1d147a80ea98: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:35:40.843689 22473 raft_consensus.cc:482] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:35:40.843858 22473 raft_consensus.cc:504] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5f5d1cbe58486589fa1d147a80ea98" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34919 } } peers { permanent_uuid: "9582eea982e04458897fd5bfaca4db9e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 39593 } } peers { permanent_uuid: "c79ddaa847094ee3bf3a8b219cf6157a" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41647 } }
I20260619 20:35:40.844277 22473 leader_election.cc:288] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5f5d1cbe58486589fa1d147a80ea98 (127.4.83.66:34919), c79ddaa847094ee3bf3a8b219cf6157a (127.4.83.65:41647)
I20260619 20:35:40.847012 22462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.844575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:58953 (local address 127.4.83.66:34919)
0619 20:35:40.844631 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:40.844633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:40.844642 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:35:40.844669 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:40.844670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:40.844677 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:40.844723 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:40.844867 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.845339 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.846269 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.846682 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.846757 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.846768 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:40.846893 (+ 125us) server_negotiation.cc:299] Negotiation successful
0619 20:35:40.846935 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 20:35:40.847047 22453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.844465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:34919 (local address 127.4.83.67:58953)
0619 20:35:40.844526 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:40.844537 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:40.844582 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:40.844738 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:40.844739 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:40.844740 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:40.844808 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:40.844810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.845415 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.845416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:40.846179 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:40.846181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.846760 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.846761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:40.846829 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.846832 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:35:40.846885 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 20:35:40.846935 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 20:35:40.847062 22461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.844538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:43041 (local address 127.4.83.65:41647)
0619 20:35:40.844585 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:40.844587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:40.844725 (+ 138us) server_negotiation.cc:407] Connection header received
0619 20:35:40.844784 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:40.844785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:40.844798 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:40.844844 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:40.844985 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.845434 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.846334 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.846748 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.846808 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.846819 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:40.846953 (+ 134us) server_negotiation.cc:299] Negotiation successful
0619 20:35:40.846992 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 20:35:40.847361 22474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.844491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:41647 (local address 127.4.83.67:43041)
0619 20:35:40.844663 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:40.844671 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:40.844724 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:40.844855 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:40.844856 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:40.844857 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:40.844926 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:40.844928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.845511 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.845512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:40.846251 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:40.846252 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.846820 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.846821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:40.846890 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.846893 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:35:40.846936 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 20:35:40.846976 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":100,"threads_started":1}
I20260619 20:35:40.847385 22274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7efc5620ae2e4f6baa16b5af96009a36" candidate_uuid: "9582eea982e04458897fd5bfaca4db9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5f5d1cbe58486589fa1d147a80ea98" is_pre_election: true
I20260619 20:35:40.847568 22274 raft_consensus.cc:2454] T 7efc5620ae2e4f6baa16b5af96009a36 P 5e5f5d1cbe58486589fa1d147a80ea98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9582eea982e04458897fd5bfaca4db9e in term 0.
I20260619 20:35:40.847786 22143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7efc5620ae2e4f6baa16b5af96009a36" candidate_uuid: "9582eea982e04458897fd5bfaca4db9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79ddaa847094ee3bf3a8b219cf6157a" is_pre_election: true
I20260619 20:35:40.847855 22338 leader_election.cc:302] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5f5d1cbe58486589fa1d147a80ea98, 9582eea982e04458897fd5bfaca4db9e; no voters:
I20260619 20:35:40.847949 22143 raft_consensus.cc:2454] T 7efc5620ae2e4f6baa16b5af96009a36 P c79ddaa847094ee3bf3a8b219cf6157a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9582eea982e04458897fd5bfaca4db9e in term 0.
I20260619 20:35:40.848026 22473 raft_consensus.cc:2781] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:35:40.848104 22473 raft_consensus.cc:482] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:35:40.848141 22473 raft_consensus.cc:3037] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:40.848848 22473 raft_consensus.cc:504] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5f5d1cbe58486589fa1d147a80ea98" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34919 } } peers { permanent_uuid: "9582eea982e04458897fd5bfaca4db9e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 39593 } } peers { permanent_uuid: "c79ddaa847094ee3bf3a8b219cf6157a" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41647 } }
I20260619 20:35:40.848989 22473 leader_election.cc:288] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e [CANDIDATE]: Term 1 election: Requested vote from peers 5e5f5d1cbe58486589fa1d147a80ea98 (127.4.83.66:34919), c79ddaa847094ee3bf3a8b219cf6157a (127.4.83.65:41647)
I20260619 20:35:40.849184 22143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7efc5620ae2e4f6baa16b5af96009a36" candidate_uuid: "9582eea982e04458897fd5bfaca4db9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79ddaa847094ee3bf3a8b219cf6157a"
I20260619 20:35:40.849275 22143 raft_consensus.cc:3037] T 7efc5620ae2e4f6baa16b5af96009a36 P c79ddaa847094ee3bf3a8b219cf6157a [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:40.849328 22274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7efc5620ae2e4f6baa16b5af96009a36" candidate_uuid: "9582eea982e04458897fd5bfaca4db9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5f5d1cbe58486589fa1d147a80ea98"
I20260619 20:35:40.849529 22274 raft_consensus.cc:3037] T 7efc5620ae2e4f6baa16b5af96009a36 P 5e5f5d1cbe58486589fa1d147a80ea98 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:40.850098 22143 raft_consensus.cc:2454] T 7efc5620ae2e4f6baa16b5af96009a36 P c79ddaa847094ee3bf3a8b219cf6157a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9582eea982e04458897fd5bfaca4db9e in term 1.
I20260619 20:35:40.850461 22340 leader_election.cc:302] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9582eea982e04458897fd5bfaca4db9e, c79ddaa847094ee3bf3a8b219cf6157a; no voters:
I20260619 20:35:40.850703 22274 raft_consensus.cc:2454] T 7efc5620ae2e4f6baa16b5af96009a36 P 5e5f5d1cbe58486589fa1d147a80ea98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9582eea982e04458897fd5bfaca4db9e in term 1.
I20260619 20:35:40.850708 22473 raft_consensus.cc:2781] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:35:40.850898 22473 raft_consensus.cc:686] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e [term 1 LEADER]: Becoming Leader. State: Replica: 9582eea982e04458897fd5bfaca4db9e, State: Running, Role: LEADER
I20260619 20:35:40.851029 22473 consensus_queue.cc:237] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5f5d1cbe58486589fa1d147a80ea98" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34919 } } peers { permanent_uuid: "9582eea982e04458897fd5bfaca4db9e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 39593 } } peers { permanent_uuid: "c79ddaa847094ee3bf3a8b219cf6157a" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41647 } }
I20260619 20:35:40.852121 22000 catalog_manager.cc:5495] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9582eea982e04458897fd5bfaca4db9e (127.4.83.67). New cstate: current_term: 1 leader_uuid: "9582eea982e04458897fd5bfaca4db9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5f5d1cbe58486589fa1d147a80ea98" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9582eea982e04458897fd5bfaca4db9e" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 39593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c79ddaa847094ee3bf3a8b219cf6157a" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41647 } health_report { overall_health: UNKNOWN } } }
I20260619 20:35:40.886983 22464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.884041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41358 (local address 127.4.83.67:39593)
0619 20:35:40.884103 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:40.884105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:40.884113 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:35:40.884161 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:40.884162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:40.884186 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:40.884252 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:35:40.884524 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.885033 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.885837 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:40.886046 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:40.886112 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:40.886509 (+ 397us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:35:40.886631 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:35:40.886717 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:40.886864 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 20:35:40.886888 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 20:35:40.889230 22274 raft_consensus.cc:1261] T 7efc5620ae2e4f6baa16b5af96009a36 P 5e5f5d1cbe58486589fa1d147a80ea98 [term 1 FOLLOWER]: Refusing update from remote peer 9582eea982e04458897fd5bfaca4db9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:35:40.889294 22143 raft_consensus.cc:1261] T 7efc5620ae2e4f6baa16b5af96009a36 P c79ddaa847094ee3bf3a8b219cf6157a [term 1 FOLLOWER]: Refusing update from remote peer 9582eea982e04458897fd5bfaca4db9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:35:40.889663 22475 consensus_queue.cc:1035] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5f5d1cbe58486589fa1d147a80ea98" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 34919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:35:40.889837 22473 consensus_queue.cc:1035] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c79ddaa847094ee3bf3a8b219cf6157a" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:35:40.893956 22480 mvcc.cc:205] Tried to move back new op lower bound from 7298667892278968320 to 7298667892126982144. Current Snapshot: MvccSnapshot[applied={T|T < 7298667892278968320 or (T in {7298667892278968320})}]
I20260619 20:35:40.895897 22001 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:35:40.896032 22001 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:35:40.896960 22001 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 57037bd30f8044839b490520717fd89c: Sending DeleteTablet for 3 replicas of tablet 7efc5620ae2e4f6baa16b5af96009a36
I20260619 20:35:40.897329 22385 tablet_service.cc:1514] Processing DeleteTablet for tablet 7efc5620ae2e4f6baa16b5af96009a36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:35:40 UTC) from {username='slave'} at 127.0.0.1:41348
I20260619 20:35:40.897409 22123 tablet_service.cc:1514] Processing DeleteTablet for tablet 7efc5620ae2e4f6baa16b5af96009a36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:35:40 UTC) from {username='slave'} at 127.0.0.1:40274
I20260619 20:35:40.897614 22487 tablet_replica.cc:330] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e: stopping tablet replica
I20260619 20:35:40.897604 22254 tablet_service.cc:1514] Processing DeleteTablet for tablet 7efc5620ae2e4f6baa16b5af96009a36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:35:40 UTC) from {username='slave'} at 127.0.0.1:36478
I20260619 20:35:40.897715 22487 raft_consensus.cc:2229] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:35:40.897830 22487 raft_consensus.cc:2258] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:35:40.897825 22488 tablet_replica.cc:330] T 7efc5620ae2e4f6baa16b5af96009a36 P c79ddaa847094ee3bf3a8b219cf6157a: stopping tablet replica
I20260619 20:35:40.897929 22488 raft_consensus.cc:2229] T 7efc5620ae2e4f6baa16b5af96009a36 P c79ddaa847094ee3bf3a8b219cf6157a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:35:40.898031 22488 raft_consensus.cc:2258] T 7efc5620ae2e4f6baa16b5af96009a36 P c79ddaa847094ee3bf3a8b219cf6157a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:35:40.898136 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 22061
I20260619 20:35:40.898412 22487 ts_tablet_manager.cc:1905] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:35:40.898733 22489 tablet_replica.cc:330] T 7efc5620ae2e4f6baa16b5af96009a36 P 5e5f5d1cbe58486589fa1d147a80ea98: stopping tablet replica
I20260619 20:35:40.898841 22489 raft_consensus.cc:2229] T 7efc5620ae2e4f6baa16b5af96009a36 P 5e5f5d1cbe58486589fa1d147a80ea98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:35:40.898937 22489 raft_consensus.cc:2258] T 7efc5620ae2e4f6baa16b5af96009a36 P 5e5f5d1cbe58486589fa1d147a80ea98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:35:40.899411 22489 ts_tablet_manager.cc:1905] T 7efc5620ae2e4f6baa16b5af96009a36 P 5e5f5d1cbe58486589fa1d147a80ea98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:35:40.900130 22487 ts_tablet_manager.cc:1918] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:35:40.900197 22487 log.cc:1196] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-2/wal/wals/7efc5620ae2e4f6baa16b5af96009a36
I20260619 20:35:40.900440 22487 ts_tablet_manager.cc:1939] T 7efc5620ae2e4f6baa16b5af96009a36 P 9582eea982e04458897fd5bfaca4db9e: Deleting consensus metadata
I20260619 20:35:40.901090 22489 ts_tablet_manager.cc:1918] T 7efc5620ae2e4f6baa16b5af96009a36 P 5e5f5d1cbe58486589fa1d147a80ea98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:35:40.901089 21985 catalog_manager.cc:4841] TS 9582eea982e04458897fd5bfaca4db9e (127.4.83.67:39593): tablet 7efc5620ae2e4f6baa16b5af96009a36 (table test-table [id=0b946861995f4deabdf55a4e913e6004]) successfully deleted
I20260619 20:35:40.901161 22489 log.cc:1196] T 7efc5620ae2e4f6baa16b5af96009a36 P 5e5f5d1cbe58486589fa1d147a80ea98: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901102012824-4429-0/minicluster-data/ts-1/wal/wals/7efc5620ae2e4f6baa16b5af96009a36
I20260619 20:35:40.901352 22489 ts_tablet_manager.cc:1939] T 7efc5620ae2e4f6baa16b5af96009a36 P 5e5f5d1cbe58486589fa1d147a80ea98: Deleting consensus metadata
I20260619 20:35:40.902354 21985 catalog_manager.cc:4841] TS 5e5f5d1cbe58486589fa1d147a80ea98 (127.4.83.66:34919): tablet 7efc5620ae2e4f6baa16b5af96009a36 (table test-table [id=0b946861995f4deabdf55a4e913e6004]) successfully deleted
W20260619 20:35:40.906810 21987 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.83.65:41647 (error 108)
I20260619 20:35:40.907033 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 22192
I20260619 20:35:40.908123 22463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:40.907971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:41647 (local address 127.0.0.1:40280)
0619 20:35:40.908025 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:40.908071 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.65:41647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 20:35:40.908252 21987 catalog_manager.cc:4584] TS c79ddaa847094ee3bf3a8b219cf6157a (127.4.83.65:41647): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7efc5620ae2e4f6baa16b5af96009a36: Network error: Client connection negotiation failed: client connection to 127.4.83.65:41647: connect: Connection refused (error 111)
I20260619 20:35:40.911404 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 22323
I20260619 20:35:40.916576 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 21970
2026-06-19T20:35:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (758 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:35:40 dist-test-slave-4m58 krb5kdc[22493](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:35:40 dist-test-slave-4m58 krb5kdc[22493](info): set up 2 sockets
Jun 19 20:35:40 dist-test-slave-4m58 krb5kdc[22493](info): commencing operation
krb5kdc: starting...
W20260619 20:35:41.884471 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.949s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:35:41 dist-test-slave-4m58 krb5kdc[22493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:35:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:35:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
I20260619 20:35:41.998780 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:45921
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:45921
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:35:42.080854 22506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:42.081115 22506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:42.081151 22506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:42.082553 22506 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:35:42.082614 22506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:42.082633 22506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:42.082649 22506 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:35:42.082664 22506 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:35:42.084379 22506 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:45921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:45921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:42.084656 22506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:42.084880 22506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:42.089713 22514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:42.089728 22512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:42.089836 22506 server_base.cc:1034] running on GCE node
W20260619 20:35:42.089728 22511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:42.090159 22506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:42.090458 22506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:42.091621 22506 hybrid_clock.cc:648] HybridClock initialized: now 1781901342091603 us; error 36 us; skew 500 ppm
Jun 19 20:35:42 dist-test-slave-4m58 krb5kdc[22493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901342, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:35:42.094164 22506 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:35:42.094623 22506 webserver.cc:460] Webserver started at http://127.4.83.126:37469/ using document root <none> and password file <none>
I20260619 20:35:42.094810 22506 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:42.094847 22506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:42.094975 22506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:42.096028 22506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "b686eff771d54ac091ffede35ab2c38b"
format_stamp: "Formatted at 2026-06-19 20:35:42 on dist-test-slave-4m58"
server_key: "34af62ce9abc717b05665f7e666135f0"
server_key_iv: "16eb8b3d5ee566b1ceab72c904eda5fe"
server_key_version: "encryptionkey@0"
I20260619 20:35:42.096313 22506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "b686eff771d54ac091ffede35ab2c38b"
format_stamp: "Formatted at 2026-06-19 20:35:42 on dist-test-slave-4m58"
server_key: "34af62ce9abc717b05665f7e666135f0"
server_key_iv: "16eb8b3d5ee566b1ceab72c904eda5fe"
server_key_version: "encryptionkey@0"
I20260619 20:35:42.097548 22506 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:42.098264 22521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:42.098467 22506 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:42.098552 22506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "b686eff771d54ac091ffede35ab2c38b"
format_stamp: "Formatted at 2026-06-19 20:35:42 on dist-test-slave-4m58"
server_key: "34af62ce9abc717b05665f7e666135f0"
server_key_iv: "16eb8b3d5ee566b1ceab72c904eda5fe"
server_key_version: "encryptionkey@0"
I20260619 20:35:42.098613 22506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:42.116376 22506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:42.118638 22506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:42.118760 22506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:42.122494 22506 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:45921
I20260619 20:35:42.122546 22573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:45921 every 8 connection(s)
I20260619 20:35:42.122881 22506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:35:42.123759 22574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:42.123991 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 22506
I20260619 20:35:42.124117 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:35:42.124315 4429 external_mini_cluster.cc:1428] Setting key 1e8548e4b0965b512f4c75544c4b1fda
I20260619 20:35:42.126412 22574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b686eff771d54ac091ffede35ab2c38b: Bootstrap starting.
I20260619 20:35:42.126966 22574 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b686eff771d54ac091ffede35ab2c38b: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:42.127241 22574 log.cc:824] T 00000000000000000000000000000000 P b686eff771d54ac091ffede35ab2c38b: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:42.127986 22574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b686eff771d54ac091ffede35ab2c38b: No bootstrap required, opened a new log
I20260619 20:35:42.130328 22574 raft_consensus.cc:348] T 00000000000000000000000000000000 P b686eff771d54ac091ffede35ab2c38b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b686eff771d54ac091ffede35ab2c38b" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45921 } }
I20260619 20:35:42.130510 22574 raft_consensus.cc:374] T 00000000000000000000000000000000 P b686eff771d54ac091ffede35ab2c38b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:42.130538 22574 raft_consensus.cc:729] T 00000000000000000000000000000000 P b686eff771d54ac091ffede35ab2c38b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b686eff771d54ac091ffede35ab2c38b, State: Initialized, Role: FOLLOWER
I20260619 20:35:42.130656 22574 consensus_queue.cc:260] T 00000000000000000000000000000000 P b686eff771d54ac091ffede35ab2c38b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b686eff771d54ac091ffede35ab2c38b" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45921 } }
I20260619 20:35:42.130726 22574 raft_consensus.cc:388] T 00000000000000000000000000000000 P b686eff771d54ac091ffede35ab2c38b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:35:42.130754 22574 raft_consensus.cc:482] T 00000000000000000000000000000000 P b686eff771d54ac091ffede35ab2c38b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:35:42.130800 22574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b686eff771d54ac091ffede35ab2c38b [term 0 FOLLOWER]: Advancing to term 1
Jun 19 20:35:42 dist-test-slave-4m58 krb5kdc[22493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:35:42.131414 22574 raft_consensus.cc:504] T 00000000000000000000000000000000 P b686eff771d54ac091ffede35ab2c38b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b686eff771d54ac091ffede35ab2c38b" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45921 } }
I20260619 20:35:42.131522 22574 leader_election.cc:302] T 00000000000000000000000000000000 P b686eff771d54ac091ffede35ab2c38b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b686eff771d54ac091ffede35ab2c38b; no voters:
I20260619 20:35:42.131704 22574 leader_election.cc:288] T 00000000000000000000000000000000 P b686eff771d54ac091ffede35ab2c38b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:35:42.131740 22579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b686eff771d54ac091ffede35ab2c38b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:35:42.131924 22579 raft_consensus.cc:686] T 00000000000000000000000000000000 P b686eff771d54ac091ffede35ab2c38b [term 1 LEADER]: Becoming Leader. State: Replica: b686eff771d54ac091ffede35ab2c38b, State: Running, Role: LEADER
I20260619 20:35:42.132014 22579 consensus_queue.cc:237] T 00000000000000000000000000000000 P b686eff771d54ac091ffede35ab2c38b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b686eff771d54ac091ffede35ab2c38b" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45921 } }
I20260619 20:35:42.132040 22574 sys_catalog.cc:564] T 00000000000000000000000000000000 P b686eff771d54ac091ffede35ab2c38b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:35:42.132326 22579 sys_catalog.cc:455] T 00000000000000000000000000000000 P b686eff771d54ac091ffede35ab2c38b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b686eff771d54ac091ffede35ab2c38b. Latest consensus state: current_term: 1 leader_uuid: "b686eff771d54ac091ffede35ab2c38b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b686eff771d54ac091ffede35ab2c38b" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45921 } } }
I20260619 20:35:42.132400 22579 sys_catalog.cc:458] T 00000000000000000000000000000000 P b686eff771d54ac091ffede35ab2c38b [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:42.132441 22580 sys_catalog.cc:455] T 00000000000000000000000000000000 P b686eff771d54ac091ffede35ab2c38b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b686eff771d54ac091ffede35ab2c38b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b686eff771d54ac091ffede35ab2c38b" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45921 } } }
I20260619 20:35:42.132544 22580 sys_catalog.cc:458] T 00000000000000000000000000000000 P b686eff771d54ac091ffede35ab2c38b [sys.catalog]: This master's current role is: LEADER
W20260619 20:35:42.133816 22594 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b686eff771d54ac091ffede35ab2c38b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:35:42.133865 22594 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:35:42.133937 22584 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:35:42.134111 22584 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:35:42.135674 22584 catalog_manager.cc:1347] Generated new cluster ID: 9529b528e6df4aa8af668b089eca698f
I20260619 20:35:42.135720 22584 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:35:42.136075 22577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:42.124846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41398 (local address 127.4.83.126:45921)
0619 20:35:42.125032 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:42.125042 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:42.125068 (+ 26us) server_negotiation.cc:407] Connection header received
0619 20:35:42.125496 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:42.125502 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:42.125547 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:42.125672 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:42.126289 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.127266 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.128306 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.128720 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.128809 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:42.131472 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:42.131477 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:42.131478 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:35:42.131492 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:42.133539 (+ 2047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:42.134737 (+ 1198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:42.134740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:42.134740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:42.134842 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:42.135059 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:42.135060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:42.135060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:42.135356 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:42.135509 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:42.135709 (+ 200us) server_negotiation.cc:299] Negotiation successful
0619 20:35:42.135875 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":88,"threads_started":1}
I20260619 20:35:42.144383 22584 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:35:42.144950 22584 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:35:42.149588 22584 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b686eff771d54ac091ffede35ab2c38b: Generated new TSK 0
I20260619 20:35:42.149760 22584 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.83.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
I20260619 20:35:42.180831 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:0
--local_ip_for_outbound_sockets=127.4.83.65
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:45921
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:35:42.267195 22600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:42.267427 22600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:42.267454 22600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:42.268769 22600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:42.268824 22600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:42.268863 22600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:35:42.270418 22600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:45921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:42.270658 22600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:42.270897 22600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:42.275991 22606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:42.275993 22608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:42.276065 22605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:42.276400 22600 server_base.cc:1034] running on GCE node
I20260619 20:35:42.276649 22600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:42.276880 22600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:42.278040 22600 hybrid_clock.cc:648] HybridClock initialized: now 1781901342278023 us; error 26 us; skew 500 ppm
Jun 19 20:35:42 dist-test-slave-4m58 krb5kdc[22493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901342, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:35:42.280834 22600 init.cc:377] Logged in from keytab as kudu/127.4.83.65@KRBTEST.COM (short username kudu)
I20260619 20:35:42.281302 22600 webserver.cc:460] Webserver started at http://127.4.83.65:44793/ using document root <none> and password file <none>
I20260619 20:35:42.281510 22600 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:42.281559 22600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:42.281688 22600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:42.282758 22600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-0/data/instance:
uuid: "1793658675934d7b8e21548a65cb1695"
format_stamp: "Formatted at 2026-06-19 20:35:42 on dist-test-slave-4m58"
server_key: "dc689f304da8f654e116a6444c34be81"
server_key_iv: "ca30963515183245ccc6ed78fd6d296d"
server_key_version: "encryptionkey@0"
I20260619 20:35:42.283090 22600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance:
uuid: "1793658675934d7b8e21548a65cb1695"
format_stamp: "Formatted at 2026-06-19 20:35:42 on dist-test-slave-4m58"
server_key: "dc689f304da8f654e116a6444c34be81"
server_key_iv: "ca30963515183245ccc6ed78fd6d296d"
server_key_version: "encryptionkey@0"
I20260619 20:35:42.284451 22600 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260619 20:35:42.285101 22615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:42.285233 22600 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:35:42.285315 22600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "1793658675934d7b8e21548a65cb1695"
format_stamp: "Formatted at 2026-06-19 20:35:42 on dist-test-slave-4m58"
server_key: "dc689f304da8f654e116a6444c34be81"
server_key_iv: "ca30963515183245ccc6ed78fd6d296d"
server_key_version: "encryptionkey@0"
I20260619 20:35:42.285374 22600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:42.295025 22600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:42.297397 22600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:42.297554 22600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:42.297782 22600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:35:42.298080 22600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:35:42.298113 22600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:42.298141 22600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:35:42.298158 22600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:42.304316 22600 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:34217
I20260619 20:35:42.304378 22728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:34217 every 8 connection(s)
I20260619 20:35:42.304689 22600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
I20260619 20:35:42.306516 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 22600
I20260619 20:35:42.306618 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance
I20260619 20:35:42.306792 4429 external_mini_cluster.cc:1428] Setting key f642b51a6782dc7ecb3c8c6e661e94ab
Jun 19 20:35:42 dist-test-slave-4m58 krb5kdc[22493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901342, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.83.66@KRBTEST.COM; defaulting to no policy
I20260619 20:35:42.315074 22577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:42.305535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:34779 (local address 127.4.83.126:45921)
0619 20:35:42.305616 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:42.305619 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:42.305764 (+ 145us) server_negotiation.cc:407] Connection header received
0619 20:35:42.306349 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:42.306350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:42.306363 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:42.306414 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:42.307514 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.308237 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.309613 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.309860 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.309934 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:42.312427 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:42.312429 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:42.312429 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:35:42.312436 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:42.313703 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:42.314090 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:42.314091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:42.314092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:42.314145 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:42.314362 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:42.314363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:42.314363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:42.314492 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:42.314549 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:42.314873 (+ 324us) server_negotiation.cc:299] Negotiation successful
0619 20:35:42.314972 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 20:35:42.315240 22731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:42.305463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:45921 (local address 127.4.83.65:34779)
0619 20:35:42.305645 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:42.305687 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:42.306219 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:42.306484 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:42.306488 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:42.307197 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:35:42.307411 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:42.307418 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.308367 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.308368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:42.309494 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:42.309497 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.309962 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.309963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:42.310106 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:42.310687 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:35:42.310697 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:35:42.312320 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:35:42.313782 (+ 1462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:35:42.313791 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:35:42.313794 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:35:42.314023 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:35:42.314199 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:35:42.314200 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:35:42.314200 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:35:42.314307 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:35:42.314547 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:35:42.314553 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:35:42.314726 (+ 173us) client_negotiation.cc:769] Sending connection context
0619 20:35:42.314829 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 20:35:42.315015 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":86,"threads_started":1}
I20260619 20:35:42.315800 22729 heartbeater.cc:347] Connected to a master server at 127.4.83.126:45921
I20260619 20:35:42.315925 22729 heartbeater.cc:464] Registering TS with master...
I20260619 20:35:42.316083 22729 heartbeater.cc:511] Master 127.4.83.126:45921 requested a full tablet report, sending...
I20260619 20:35:42.316608 22538 ts_manager.cc:194] Registered new tserver with Master: 1793658675934d7b8e21548a65cb1695 (127.4.83.65:34217)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.66@KRBTEST.COM" created.
I20260619 20:35:42.317333 22538 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.65@KRBTEST.COM'} at 127.4.83.65:34779
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
I20260619 20:35:42.330891 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:0
--local_ip_for_outbound_sockets=127.4.83.66
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:45921
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:35:42.416251 22734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:42.416467 22734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:42.416493 22734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:42.417644 22734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:42.417685 22734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:42.417711 22734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:35:42.419337 22734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:45921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:42.419574 22734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:42.419775 22734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:42.425527 22742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:42.425498 22740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:42.425509 22739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:42.425734 22734 server_base.cc:1034] running on GCE node
I20260619 20:35:42.426003 22734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:42.426226 22734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:42.427361 22734 hybrid_clock.cc:648] HybridClock initialized: now 1781901342427348 us; error 30 us; skew 500 ppm
Jun 19 20:35:42 dist-test-slave-4m58 krb5kdc[22493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901342, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:35:42.429898 22734 init.cc:377] Logged in from keytab as kudu/127.4.83.66@KRBTEST.COM (short username kudu)
I20260619 20:35:42.430334 22734 webserver.cc:460] Webserver started at http://127.4.83.66:40653/ using document root <none> and password file <none>
I20260619 20:35:42.430548 22734 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:42.430596 22734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:42.430721 22734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:42.431788 22734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-1/data/instance:
uuid: "78a81afa549640a4a56152e28f606070"
format_stamp: "Formatted at 2026-06-19 20:35:42 on dist-test-slave-4m58"
server_key: "f1ebae3556ed9a8e6eb0cf695f1f17e8"
server_key_iv: "4e337a0413bde27d19b9377f71074df8"
server_key_version: "encryptionkey@0"
I20260619 20:35:42.432251 22734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance:
uuid: "78a81afa549640a4a56152e28f606070"
format_stamp: "Formatted at 2026-06-19 20:35:42 on dist-test-slave-4m58"
server_key: "f1ebae3556ed9a8e6eb0cf695f1f17e8"
server_key_iv: "4e337a0413bde27d19b9377f71074df8"
server_key_version: "encryptionkey@0"
I20260619 20:35:42.433660 22734 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:42.434466 22749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:42.434675 22734 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:42.434764 22734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "78a81afa549640a4a56152e28f606070"
format_stamp: "Formatted at 2026-06-19 20:35:42 on dist-test-slave-4m58"
server_key: "f1ebae3556ed9a8e6eb0cf695f1f17e8"
server_key_iv: "4e337a0413bde27d19b9377f71074df8"
server_key_version: "encryptionkey@0"
I20260619 20:35:42.434857 22734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:42.447669 22734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:42.449829 22734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:42.450033 22734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:42.450273 22734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:35:42.450592 22734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:35:42.450626 22734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:42.450657 22734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:35:42.450675 22734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:42.457720 22734 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:40151
I20260619 20:35:42.457897 22862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:40151 every 8 connection(s)
I20260619 20:35:42.458195 22734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
Jun 19 20:35:42 dist-test-slave-4m58 krb5kdc[22493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901342, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:35:42.465926 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 22734
I20260619 20:35:42.466037 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance
I20260619 20:35:42.466217 4429 external_mini_cluster.cc:1428] Setting key dbc1841f7cc7b0a4449ae54375353dc2
I20260619 20:35:42.468742 22577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:42.458971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:34557 (local address 127.4.83.126:45921)
0619 20:35:42.459035 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:42.459037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:42.459221 (+ 184us) server_negotiation.cc:407] Connection header received
0619 20:35:42.459923 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:42.459924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:42.459936 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:42.460000 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:42.461275 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.462045 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.463338 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.463633 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.463696 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:42.465864 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:42.465865 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:42.465865 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:35:42.465873 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:42.467295 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:42.467693 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:42.467694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:42.467694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:42.467757 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:42.468040 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:42.468040 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:42.468040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:42.468189 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:42.468251 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:42.468593 (+ 342us) server_negotiation.cc:299] Negotiation successful
0619 20:35:42.468670 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 20:35:42.469012 22865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:42.458924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:45921 (local address 127.4.83.66:34557)
0619 20:35:42.459093 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:42.459142 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:42.459801 (+ 659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:42.460076 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:42.460081 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:42.460865 (+ 784us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:35:42.461161 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:42.461167 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.462135 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.462136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:42.463243 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:42.463247 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.463720 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.463721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:42.463891 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:42.464350 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:35:42.464358 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:35:42.465770 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:35:42.467378 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:35:42.467386 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:35:42.467388 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:35:42.467624 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:35:42.467813 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:35:42.467813 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:35:42.467813 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:35:42.467984 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:35:42.468255 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:35:42.468261 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:35:42.468452 (+ 191us) client_negotiation.cc:769] Sending connection context
0619 20:35:42.468557 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 20:35:42.468738 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":78,"threads_started":1}
I20260619 20:35:42.469471 22863 heartbeater.cc:347] Connected to a master server at 127.4.83.126:45921
I20260619 20:35:42.469565 22863 heartbeater.cc:464] Registering TS with master...
I20260619 20:35:42.469745 22863 heartbeater.cc:511] Master 127.4.83.126:45921 requested a full tablet report, sending...
I20260619 20:35:42.470129 22538 ts_manager.cc:194] Registered new tserver with Master: 78a81afa549640a4a56152e28f606070 (127.4.83.66:40151)
I20260619 20:35:42.470481 22538 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.66@KRBTEST.COM'} at 127.4.83.66:34557
WARNING: no policy specified for kudu/127.4.83.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
I20260619 20:35:42.490857 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:0
--local_ip_for_outbound_sockets=127.4.83.67
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:45921
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:35:42.572134 22868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:42.572374 22868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:42.572402 22868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:42.573658 22868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:42.573714 22868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:42.573742 22868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:35:42.575330 22868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:45921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:42.575584 22868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:42.575824 22868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:35:42.580900 22868 server_base.cc:1034] running on GCE node
W20260619 20:35:42.580857 22874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:42.580894 22873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:42.580837 22876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:42.581280 22868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:42.581534 22868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:42.582688 22868 hybrid_clock.cc:648] HybridClock initialized: now 1781901342582675 us; error 27 us; skew 500 ppm
Jun 19 20:35:42 dist-test-slave-4m58 krb5kdc[22493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901342, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:35:42.585239 22868 init.cc:377] Logged in from keytab as kudu/127.4.83.67@KRBTEST.COM (short username kudu)
I20260619 20:35:42.585688 22868 webserver.cc:460] Webserver started at http://127.4.83.67:36337/ using document root <none> and password file <none>
I20260619 20:35:42.585925 22868 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:42.585974 22868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:42.586097 22868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:42.587132 22868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-2/data/instance:
uuid: "23db05ba734b48578946a450987351eb"
format_stamp: "Formatted at 2026-06-19 20:35:42 on dist-test-slave-4m58"
server_key: "7ddb6502bd0608c2f6e523be4116cc2c"
server_key_iv: "55949af554913b8fb5e80dc4f0943553"
server_key_version: "encryptionkey@0"
I20260619 20:35:42.587491 22868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance:
uuid: "23db05ba734b48578946a450987351eb"
format_stamp: "Formatted at 2026-06-19 20:35:42 on dist-test-slave-4m58"
server_key: "7ddb6502bd0608c2f6e523be4116cc2c"
server_key_iv: "55949af554913b8fb5e80dc4f0943553"
server_key_version: "encryptionkey@0"
I20260619 20:35:42.588901 22868 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:35:42.589605 22883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:42.589779 22868 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:35:42.589898 22868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "23db05ba734b48578946a450987351eb"
format_stamp: "Formatted at 2026-06-19 20:35:42 on dist-test-slave-4m58"
server_key: "7ddb6502bd0608c2f6e523be4116cc2c"
server_key_iv: "55949af554913b8fb5e80dc4f0943553"
server_key_version: "encryptionkey@0"
I20260619 20:35:42.589965 22868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:42.603730 22868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:42.605852 22868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:42.606021 22868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:42.606262 22868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:35:42.606567 22868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:35:42.606602 22868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:42.606632 22868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:35:42.606655 22868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:42.612493 22868 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:39925
I20260619 20:35:42.612538 22996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:39925 every 8 connection(s)
I20260619 20:35:42.612854 22868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
I20260619 20:35:42.615751 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 22868
I20260619 20:35:42.615901 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance
I20260619 20:35:42.616055 4429 external_mini_cluster.cc:1428] Setting key 57f14f28972c22e8dccf09946b3ce606
Jun 19 20:35:42 dist-test-slave-4m58 krb5kdc[22493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901342, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:35:42.622040 22577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:42.613620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:37327 (local address 127.4.83.126:45921)
0619 20:35:42.613682 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:42.613684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:42.613888 (+ 204us) server_negotiation.cc:407] Connection header received
0619 20:35:42.614378 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:42.614379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:42.614391 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:42.614453 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:42.615418 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.616025 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.617241 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.617457 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.617512 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:42.619581 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:42.619583 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:42.619583 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:35:42.619591 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:42.620833 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:42.621249 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:42.621250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:42.621251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:42.621312 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:42.621543 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:42.621543 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:42.621543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:42.621652 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:42.621698 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:42.621898 (+ 200us) server_negotiation.cc:299] Negotiation successful
0619 20:35:42.621975 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 20:35:42.622247 22999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:42.613595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:45921 (local address 127.4.83.67:37327)
0619 20:35:42.613763 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:42.613807 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:42.614268 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:42.614508 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:42.614511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:42.615110 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:35:42.615301 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:42.615306 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.616116 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.616118 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:42.617162 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:42.617166 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.617529 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.617530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:42.617658 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:42.618121 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:35:42.618129 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:35:42.619484 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:35:42.620907 (+ 1423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:35:42.620913 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:35:42.620916 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:35:42.621160 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:35:42.621386 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:35:42.621387 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:35:42.621387 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:35:42.621487 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:35:42.621695 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:35:42.621698 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:35:42.621804 (+ 106us) client_negotiation.cc:769] Sending connection context
0619 20:35:42.621878 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 20:35:42.622029 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":85,"threads_started":1}
I20260619 20:35:42.622673 22997 heartbeater.cc:347] Connected to a master server at 127.4.83.126:45921
I20260619 20:35:42.622766 22997 heartbeater.cc:464] Registering TS with master...
I20260619 20:35:42.622936 22997 heartbeater.cc:511] Master 127.4.83.126:45921 requested a full tablet report, sending...
I20260619 20:35:42.623289 22538 ts_manager.cc:194] Registered new tserver with Master: 23db05ba734b48578946a450987351eb (127.4.83.67:39925)
I20260619 20:35:42.623638 22538 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.67@KRBTEST.COM'} at 127.4.83.67:37327
I20260619 20:35:42.627713 4429 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:35:42.635305 22577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:42.629121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41402 (local address 127.4.83.126:45921)
0619 20:35:42.629207 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:42.629209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:42.629285 (+ 76us) server_negotiation.cc:407] Connection header received
0619 20:35:42.629343 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:42.629343 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:42.629353 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:42.629406 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:42.629922 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.630430 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.631245 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.631448 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.631510 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:42.632907 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:42.632908 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:42.632908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:35:42.632915 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:42.634229 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:42.634599 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:42.634599 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:42.634600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:42.634655 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:42.634823 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:42.634824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:42.634824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:42.634959 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:42.635045 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:42.635169 (+ 124us) server_negotiation.cc:299] Negotiation successful
0619 20:35:42.635216 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 20:35:42.636924 22538 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41402:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:35:42.638553 22538 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:35:42.644946 23009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:42.641129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46942 (local address 127.4.83.65:34217)
0619 20:35:42.641304 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:42.641309 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:42.641319 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:35:42.641374 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:42.641376 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:42.641392 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:42.641659 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:42.641728 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.642843 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.644081 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.644551 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.644613 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:42.644674 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:42.644819 (+ 145us) server_negotiation.cc:299] Negotiation successful
0619 20:35:42.644860 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":78,"threads_started":1}
I20260619 20:35:42.644997 23008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:42.641013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:40151 (local address 127.0.0.1:60652)
0619 20:35:42.641415 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:42.641423 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:42.641467 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:42.641704 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:42.641705 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:42.641706 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:42.641747 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:42.641748 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.642591 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.642592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:42.643741 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:42.643744 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.644743 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.644744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:42.644817 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:42.644826 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 20:35:42.644866 (+ 40us) client_negotiation.cc:240] Negotiation successful
0619 20:35:42.644917 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"mutex_wait_us":2,"thread_start_us":40,"threads_started":1}
I20260619 20:35:42.645131 23005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:42.640784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:34217 (local address 127.0.0.1:46942)
0619 20:35:42.640891 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:42.640931 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:42.640997 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:42.641500 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:42.641503 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:42.641506 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:42.641587 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:42.641589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.642956 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.642957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:42.644014 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:42.644016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.644630 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.644630 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:42.644723 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:42.644737 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 20:35:42.644799 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 20:35:42.644858 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"mutex_wait_us":27,"thread_start_us":54,"threads_started":1}
I20260619 20:35:42.645153 23007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:42.640936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42964 (local address 127.4.83.67:39925)
0619 20:35:42.641325 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:42.641329 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:42.641354 (+ 25us) server_negotiation.cc:407] Connection header received
0619 20:35:42.641448 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:42.641451 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:42.641466 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:42.641519 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:42.641730 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.642874 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.643969 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.644640 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.644676 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:42.644718 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:42.645033 (+ 315us) server_negotiation.cc:299] Negotiation successful
0619 20:35:42.645078 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":75,"threads_started":1}
I20260619 20:35:42.644997 23006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:42.640785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:39925 (local address 127.0.0.1:42964)
0619 20:35:42.641294 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:42.641305 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:42.641349 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:42.641614 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:42.641614 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:42.641616 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:42.641668 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:42.641670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.642973 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.642974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:42.643882 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:42.643885 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.644769 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.644769 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:42.644817 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:42.644823 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 20:35:42.644865 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 20:35:42.644907 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"mutex_wait_us":2,"thread_start_us":65,"threads_started":1}
I20260619 20:35:42.645291 23010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:42.641138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60652 (local address 127.4.83.66:40151)
0619 20:35:42.641309 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:42.641312 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:42.641520 (+ 208us) server_negotiation.cc:407] Connection header received
0619 20:35:42.641559 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:42.641561 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:42.641575 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:42.641638 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:42.641809 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.642505 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.643823 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.644576 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.644644 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:42.644704 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:42.645169 (+ 465us) server_negotiation.cc:299] Negotiation successful
0619 20:35:42.645216 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":81,"threads_started":1}
I20260619 20:35:42.646005 22663 tablet_service.cc:1467] Processing CreateTablet for tablet 16e41af8449b4d7c9e7d8f91682d04ed (DEFAULT_TABLE table=test-table [id=d5061d9e1ecc4dfd9ca2acaebc601257]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:35:42.646085 22931 tablet_service.cc:1467] Processing CreateTablet for tablet 16e41af8449b4d7c9e7d8f91682d04ed (DEFAULT_TABLE table=test-table [id=d5061d9e1ecc4dfd9ca2acaebc601257]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:35:42.646082 22797 tablet_service.cc:1467] Processing CreateTablet for tablet 16e41af8449b4d7c9e7d8f91682d04ed (DEFAULT_TABLE table=test-table [id=d5061d9e1ecc4dfd9ca2acaebc601257]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:35:42.646341 22663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16e41af8449b4d7c9e7d8f91682d04ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:42.646409 22797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16e41af8449b4d7c9e7d8f91682d04ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:42.646402 22931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16e41af8449b4d7c9e7d8f91682d04ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:42.648931 23011 tablet_bootstrap.cc:492] T 16e41af8449b4d7c9e7d8f91682d04ed P 1793658675934d7b8e21548a65cb1695: Bootstrap starting.
I20260619 20:35:42.649438 23012 tablet_bootstrap.cc:492] T 16e41af8449b4d7c9e7d8f91682d04ed P 78a81afa549640a4a56152e28f606070: Bootstrap starting.
I20260619 20:35:42.649655 23011 tablet_bootstrap.cc:654] T 16e41af8449b4d7c9e7d8f91682d04ed P 1793658675934d7b8e21548a65cb1695: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:42.650092 23012 tablet_bootstrap.cc:654] T 16e41af8449b4d7c9e7d8f91682d04ed P 78a81afa549640a4a56152e28f606070: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:42.650226 23011 log.cc:824] T 16e41af8449b4d7c9e7d8f91682d04ed P 1793658675934d7b8e21548a65cb1695: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:42.650607 23012 log.cc:824] T 16e41af8449b4d7c9e7d8f91682d04ed P 78a81afa549640a4a56152e28f606070: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:42.650913 23013 tablet_bootstrap.cc:492] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb: Bootstrap starting.
I20260619 20:35:42.650985 23011 tablet_bootstrap.cc:492] T 16e41af8449b4d7c9e7d8f91682d04ed P 1793658675934d7b8e21548a65cb1695: No bootstrap required, opened a new log
I20260619 20:35:42.651044 23011 ts_tablet_manager.cc:1397] T 16e41af8449b4d7c9e7d8f91682d04ed P 1793658675934d7b8e21548a65cb1695: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:35:42.651230 23012 tablet_bootstrap.cc:492] T 16e41af8449b4d7c9e7d8f91682d04ed P 78a81afa549640a4a56152e28f606070: No bootstrap required, opened a new log
I20260619 20:35:42.651294 23012 ts_tablet_manager.cc:1397] T 16e41af8449b4d7c9e7d8f91682d04ed P 78a81afa549640a4a56152e28f606070: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:35:42.651506 23013 tablet_bootstrap.cc:654] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:42.652104 23013 log.cc:824] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:42.652866 23013 tablet_bootstrap.cc:492] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb: No bootstrap required, opened a new log
I20260619 20:35:42.652936 23013 ts_tablet_manager.cc:1397] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:35:42.653033 23012 raft_consensus.cc:348] T 16e41af8449b4d7c9e7d8f91682d04ed P 78a81afa549640a4a56152e28f606070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1793658675934d7b8e21548a65cb1695" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34217 } } peers { permanent_uuid: "23db05ba734b48578946a450987351eb" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 39925 } } peers { permanent_uuid: "78a81afa549640a4a56152e28f606070" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40151 } }
I20260619 20:35:42.653043 23011 raft_consensus.cc:348] T 16e41af8449b4d7c9e7d8f91682d04ed P 1793658675934d7b8e21548a65cb1695 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1793658675934d7b8e21548a65cb1695" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34217 } } peers { permanent_uuid: "23db05ba734b48578946a450987351eb" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 39925 } } peers { permanent_uuid: "78a81afa549640a4a56152e28f606070" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40151 } }
I20260619 20:35:42.653185 23012 raft_consensus.cc:374] T 16e41af8449b4d7c9e7d8f91682d04ed P 78a81afa549640a4a56152e28f606070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:42.653206 23011 raft_consensus.cc:374] T 16e41af8449b4d7c9e7d8f91682d04ed P 1793658675934d7b8e21548a65cb1695 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:42.653219 23012 raft_consensus.cc:729] T 16e41af8449b4d7c9e7d8f91682d04ed P 78a81afa549640a4a56152e28f606070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78a81afa549640a4a56152e28f606070, State: Initialized, Role: FOLLOWER
I20260619 20:35:42.653249 23011 raft_consensus.cc:729] T 16e41af8449b4d7c9e7d8f91682d04ed P 1793658675934d7b8e21548a65cb1695 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1793658675934d7b8e21548a65cb1695, State: Initialized, Role: FOLLOWER
I20260619 20:35:42.653359 23011 consensus_queue.cc:260] T 16e41af8449b4d7c9e7d8f91682d04ed P 1793658675934d7b8e21548a65cb1695 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1793658675934d7b8e21548a65cb1695" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34217 } } peers { permanent_uuid: "23db05ba734b48578946a450987351eb" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 39925 } } peers { permanent_uuid: "78a81afa549640a4a56152e28f606070" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40151 } }
I20260619 20:35:42.653362 23012 consensus_queue.cc:260] T 16e41af8449b4d7c9e7d8f91682d04ed P 78a81afa549640a4a56152e28f606070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1793658675934d7b8e21548a65cb1695" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34217 } } peers { permanent_uuid: "23db05ba734b48578946a450987351eb" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 39925 } } peers { permanent_uuid: "78a81afa549640a4a56152e28f606070" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40151 } }
I20260619 20:35:42.653616 23012 ts_tablet_manager.cc:1428] T 16e41af8449b4d7c9e7d8f91682d04ed P 78a81afa549640a4a56152e28f606070: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 20:35:42.653625 23011 ts_tablet_manager.cc:1428] T 16e41af8449b4d7c9e7d8f91682d04ed P 1793658675934d7b8e21548a65cb1695: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:35:42.653650 22729 heartbeater.cc:503] Master 127.4.83.126:45921 was elected leader, sending a full tablet report...
I20260619 20:35:42.653717 22863 heartbeater.cc:503] Master 127.4.83.126:45921 was elected leader, sending a full tablet report...
I20260619 20:35:42.654838 23013 raft_consensus.cc:348] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1793658675934d7b8e21548a65cb1695" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34217 } } peers { permanent_uuid: "23db05ba734b48578946a450987351eb" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 39925 } } peers { permanent_uuid: "78a81afa549640a4a56152e28f606070" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40151 } }
I20260619 20:35:42.654986 23013 raft_consensus.cc:374] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:42.655017 23013 raft_consensus.cc:729] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23db05ba734b48578946a450987351eb, State: Initialized, Role: FOLLOWER
I20260619 20:35:42.655124 23013 consensus_queue.cc:260] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1793658675934d7b8e21548a65cb1695" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34217 } } peers { permanent_uuid: "23db05ba734b48578946a450987351eb" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 39925 } } peers { permanent_uuid: "78a81afa549640a4a56152e28f606070" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40151 } }
I20260619 20:35:42.655382 23013 ts_tablet_manager.cc:1428] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 20:35:42.655416 22997 heartbeater.cc:503] Master 127.4.83.126:45921 was elected leader, sending a full tablet report...
W20260619 20:35:42.708813 22864 tablet.cc:2369] T 16e41af8449b4d7c9e7d8f91682d04ed P 78a81afa549640a4a56152e28f606070: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:35:42.805438 22730 tablet.cc:2369] T 16e41af8449b4d7c9e7d8f91682d04ed P 1793658675934d7b8e21548a65cb1695: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:35:42.820091 23019 raft_consensus.cc:482] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:35:42.820261 23019 raft_consensus.cc:504] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1793658675934d7b8e21548a65cb1695" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34217 } } peers { permanent_uuid: "23db05ba734b48578946a450987351eb" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 39925 } } peers { permanent_uuid: "78a81afa549640a4a56152e28f606070" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40151 } }
I20260619 20:35:42.820660 23019 leader_election.cc:288] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1793658675934d7b8e21548a65cb1695 (127.4.83.65:34217), 78a81afa549640a4a56152e28f606070 (127.4.83.66:40151)
I20260619 20:35:42.824985 23010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:42.821160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:46639 (local address 127.4.83.66:40151)
0619 20:35:42.821207 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:42.821208 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:42.821337 (+ 129us) server_negotiation.cc:407] Connection header received
0619 20:35:42.821368 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:42.821369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:42.821382 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:42.821444 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:42.821624 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.822304 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.823927 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.824521 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.824594 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:42.824606 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:42.824873 (+ 267us) server_negotiation.cc:299] Negotiation successful
0619 20:35:42.824921 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 20:35:42.824965 22999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:42.820921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:34217 (local address 127.4.83.67:48365)
0619 20:35:42.820976 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:42.820989 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:42.821037 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:42.821356 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:42.821357 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:42.821359 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:42.821432 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:42.821434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.822399 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.822400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:42.823780 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:42.823783 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.824633 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.824634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:42.824728 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:42.824735 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 20:35:42.824807 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 20:35:42.824858 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":84}
I20260619 20:35:42.825001 23009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:42.821103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:48365 (local address 127.4.83.65:34217)
0619 20:35:42.821152 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:42.821154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:42.821164 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:35:42.821196 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:42.821197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:42.821209 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:42.821345 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:42.821510 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.822275 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.823943 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.824518 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.824594 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:42.824606 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:42.824845 (+ 239us) server_negotiation.cc:299] Negotiation successful
0619 20:35:42.824898 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 20:35:42.824973 23020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:42.821064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:40151 (local address 127.4.83.67:46639)
0619 20:35:42.821240 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:42.821248 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:42.821293 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:42.821455 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:42.821456 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:42.821458 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:42.821536 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:42.821538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.822399 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.822400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:42.823780 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:42.823783 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.824631 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.824632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:42.824728 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:42.824735 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 20:35:42.824856 (+ 121us) client_negotiation.cc:240] Negotiation successful
0619 20:35:42.824890 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":44,"thread_start_us":124,"threads_started":1}
I20260619 20:35:42.825421 22683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16e41af8449b4d7c9e7d8f91682d04ed" candidate_uuid: "23db05ba734b48578946a450987351eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1793658675934d7b8e21548a65cb1695" is_pre_election: true
I20260619 20:35:42.825421 22817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16e41af8449b4d7c9e7d8f91682d04ed" candidate_uuid: "23db05ba734b48578946a450987351eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78a81afa549640a4a56152e28f606070" is_pre_election: true
I20260619 20:35:42.825598 22817 raft_consensus.cc:2454] T 16e41af8449b4d7c9e7d8f91682d04ed P 78a81afa549640a4a56152e28f606070 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23db05ba734b48578946a450987351eb in term 0.
I20260619 20:35:42.825598 22683 raft_consensus.cc:2454] T 16e41af8449b4d7c9e7d8f91682d04ed P 1793658675934d7b8e21548a65cb1695 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23db05ba734b48578946a450987351eb in term 0.
I20260619 20:35:42.825846 22884 leader_election.cc:302] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23db05ba734b48578946a450987351eb, 78a81afa549640a4a56152e28f606070; no voters:
I20260619 20:35:42.826031 23019 raft_consensus.cc:2781] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:35:42.826095 23019 raft_consensus.cc:482] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:35:42.826118 23019 raft_consensus.cc:3037] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:42.826805 23019 raft_consensus.cc:504] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1793658675934d7b8e21548a65cb1695" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34217 } } peers { permanent_uuid: "23db05ba734b48578946a450987351eb" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 39925 } } peers { permanent_uuid: "78a81afa549640a4a56152e28f606070" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40151 } }
I20260619 20:35:42.826920 23019 leader_election.cc:288] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb [CANDIDATE]: Term 1 election: Requested vote from peers 1793658675934d7b8e21548a65cb1695 (127.4.83.65:34217), 78a81afa549640a4a56152e28f606070 (127.4.83.66:40151)
I20260619 20:35:42.827116 22683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16e41af8449b4d7c9e7d8f91682d04ed" candidate_uuid: "23db05ba734b48578946a450987351eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1793658675934d7b8e21548a65cb1695"
I20260619 20:35:42.827185 22683 raft_consensus.cc:3037] T 16e41af8449b4d7c9e7d8f91682d04ed P 1793658675934d7b8e21548a65cb1695 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:42.827154 22817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16e41af8449b4d7c9e7d8f91682d04ed" candidate_uuid: "23db05ba734b48578946a450987351eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78a81afa549640a4a56152e28f606070"
I20260619 20:35:42.827245 22817 raft_consensus.cc:3037] T 16e41af8449b4d7c9e7d8f91682d04ed P 78a81afa549640a4a56152e28f606070 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:42.827906 22683 raft_consensus.cc:2454] T 16e41af8449b4d7c9e7d8f91682d04ed P 1793658675934d7b8e21548a65cb1695 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23db05ba734b48578946a450987351eb in term 1.
I20260619 20:35:42.827989 22817 raft_consensus.cc:2454] T 16e41af8449b4d7c9e7d8f91682d04ed P 78a81afa549640a4a56152e28f606070 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23db05ba734b48578946a450987351eb in term 1.
I20260619 20:35:42.828107 22884 leader_election.cc:302] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1793658675934d7b8e21548a65cb1695, 23db05ba734b48578946a450987351eb; no voters:
I20260619 20:35:42.828233 23019 raft_consensus.cc:2781] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:35:42.828418 23019 raft_consensus.cc:686] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb [term 1 LEADER]: Becoming Leader. State: Replica: 23db05ba734b48578946a450987351eb, State: Running, Role: LEADER
I20260619 20:35:42.828531 23019 consensus_queue.cc:237] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1793658675934d7b8e21548a65cb1695" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34217 } } peers { permanent_uuid: "23db05ba734b48578946a450987351eb" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 39925 } } peers { permanent_uuid: "78a81afa549640a4a56152e28f606070" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40151 } }
I20260619 20:35:42.829393 22537 catalog_manager.cc:5495] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 23db05ba734b48578946a450987351eb (127.4.83.67). New cstate: current_term: 1 leader_uuid: "23db05ba734b48578946a450987351eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1793658675934d7b8e21548a65cb1695" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23db05ba734b48578946a450987351eb" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 39925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78a81afa549640a4a56152e28f606070" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40151 } health_report { overall_health: UNKNOWN } } }
W20260619 20:35:42.863442 22998 tablet.cc:2369] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:35:42.871614 23007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:42.869304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42972 (local address 127.4.83.67:39925)
0619 20:35:42.869367 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:42.869369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:42.869377 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:35:42.869412 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:42.869413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:42.869434 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:42.869500 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:35:42.869599 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.870085 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.870826 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:42.871031 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:42.871087 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:42.871214 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:35:42.871300 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:35:42.871375 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:42.871538 (+ 163us) server_negotiation.cc:299] Negotiation successful
0619 20:35:42.871549 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 20:35:42.873378 22683 raft_consensus.cc:1261] T 16e41af8449b4d7c9e7d8f91682d04ed P 1793658675934d7b8e21548a65cb1695 [term 1 FOLLOWER]: Refusing update from remote peer 23db05ba734b48578946a450987351eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:35:42.873363 22817 raft_consensus.cc:1261] T 16e41af8449b4d7c9e7d8f91682d04ed P 78a81afa549640a4a56152e28f606070 [term 1 FOLLOWER]: Refusing update from remote peer 23db05ba734b48578946a450987351eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:35:42.873724 23019 consensus_queue.cc:1035] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1793658675934d7b8e21548a65cb1695" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:35:42.873863 23021 consensus_queue.cc:1035] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "78a81afa549640a4a56152e28f606070" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 40151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:35:42.876356 23026 mvcc.cc:205] Tried to move back new op lower bound from 7298667900406837248 to 7298667900226367488. Current Snapshot: MvccSnapshot[applied={T|T < 7298667900406837248}]
I20260619 20:35:42.876649 23025 mvcc.cc:205] Tried to move back new op lower bound from 7298667900406837248 to 7298667900226367488. Current Snapshot: MvccSnapshot[applied={T|T < 7298667900406837248}]
I20260619 20:35:42.879269 22538 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:35:42.879364 22538 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:35:42.880221 22538 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b686eff771d54ac091ffede35ab2c38b: Sending DeleteTablet for 3 replicas of tablet 16e41af8449b4d7c9e7d8f91682d04ed
I20260619 20:35:42.880568 22663 tablet_service.cc:1514] Processing DeleteTablet for tablet 16e41af8449b4d7c9e7d8f91682d04ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:35:42 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:46942
I20260619 20:35:42.880580 22797 tablet_service.cc:1514] Processing DeleteTablet for tablet 16e41af8449b4d7c9e7d8f91682d04ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:35:42 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:60652
I20260619 20:35:42.880844 22931 tablet_service.cc:1514] Processing DeleteTablet for tablet 16e41af8449b4d7c9e7d8f91682d04ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:35:42 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:42964
I20260619 20:35:42.880878 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 22600
I20260619 20:35:42.881376 23034 tablet_replica.cc:330] T 16e41af8449b4d7c9e7d8f91682d04ed P 78a81afa549640a4a56152e28f606070: stopping tablet replica
I20260619 20:35:42.881527 23034 raft_consensus.cc:2229] T 16e41af8449b4d7c9e7d8f91682d04ed P 78a81afa549640a4a56152e28f606070 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:35:42.881708 23034 raft_consensus.cc:2258] T 16e41af8449b4d7c9e7d8f91682d04ed P 78a81afa549640a4a56152e28f606070 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:35:42.882248 23034 ts_tablet_manager.cc:1905] T 16e41af8449b4d7c9e7d8f91682d04ed P 78a81afa549640a4a56152e28f606070: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:35:42.882854 23036 tablet_replica.cc:330] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb: stopping tablet replica
I20260619 20:35:42.882969 23036 raft_consensus.cc:2229] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:35:42.883112 23036 raft_consensus.cc:2258] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:35:42.883595 23036 ts_tablet_manager.cc:1905] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:35:42.884177 23034 ts_tablet_manager.cc:1918] T 16e41af8449b4d7c9e7d8f91682d04ed P 78a81afa549640a4a56152e28f606070: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:35:42.884248 23034 log.cc:1196] T 16e41af8449b4d7c9e7d8f91682d04ed P 78a81afa549640a4a56152e28f606070: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-1/wal/wals/16e41af8449b4d7c9e7d8f91682d04ed
I20260619 20:35:42.884478 23034 ts_tablet_manager.cc:1939] T 16e41af8449b4d7c9e7d8f91682d04ed P 78a81afa549640a4a56152e28f606070: Deleting consensus metadata
I20260619 20:35:42.884856 22522 catalog_manager.cc:4841] TS 78a81afa549640a4a56152e28f606070 (127.4.83.66:40151): tablet 16e41af8449b4d7c9e7d8f91682d04ed (table test-table [id=d5061d9e1ecc4dfd9ca2acaebc601257]) successfully deleted
I20260619 20:35:42.885239 23036 ts_tablet_manager.cc:1918] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:35:42.885326 23036 log.cc:1196] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901102012824-4429-0/minicluster-data/ts-2/wal/wals/16e41af8449b4d7c9e7d8f91682d04ed
I20260619 20:35:42.885587 23036 ts_tablet_manager.cc:1939] T 16e41af8449b4d7c9e7d8f91682d04ed P 23db05ba734b48578946a450987351eb: Deleting consensus metadata
I20260619 20:35:42.886013 22523 catalog_manager.cc:4841] TS 23db05ba734b48578946a450987351eb (127.4.83.67:39925): tablet 16e41af8449b4d7c9e7d8f91682d04ed (table test-table [id=d5061d9e1ecc4dfd9ca2acaebc601257]) successfully deleted
W20260619 20:35:42.889755 22522 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.83.65:34217 (error 108)
I20260619 20:35:42.890039 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 22734
I20260619 20:35:42.890990 23005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:42.890197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:34217 (local address 127.0.0.1:46956)
0619 20:35:42.890904 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:42.890951 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.65:34217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":686}
W20260619 20:35:42.891956 22522 catalog_manager.cc:4584] TS 1793658675934d7b8e21548a65cb1695 (127.4.83.65:34217): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 16e41af8449b4d7c9e7d8f91682d04ed: Network error: Client connection negotiation failed: client connection to 127.4.83.65:34217: connect: Connection refused (error 111)
I20260619 20:35:42.895063 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 22868
I20260619 20:35:42.899238 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 22506
2026-06-19T20:35:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1983 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-19T20:35:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:35:42Z Disabled control of system clock
I20260619 20:35:42.925351 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:34423
--webserver_interface=127.4.83.126
--webserver_port=0
--builtin_ntp_servers=127.4.83.84:44009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:34423
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 20:35:43.005019 23041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:43.005249 23041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:43.005270 23041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:43.006398 23041 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:35:43.006441 23041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:43.006453 23041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:43.006464 23041 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:35:43.006474 23041 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:35:43.007761 23041 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:44009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:34423
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:34423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:43.007985 23041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:43.008232 23041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:43.013227 23049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:43.013212 23046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:43.013304 23041 server_base.cc:1034] running on GCE node
W20260619 20:35:43.013212 23047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:43.013692 23041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:43.014027 23041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:43.015187 23041 hybrid_clock.cc:648] HybridClock initialized: now 1781901343015163 us; error 52 us; skew 500 ppm
I20260619 20:35:43.020090 23041 webserver.cc:460] Webserver started at http://127.4.83.126:35823/ using document root <none> and password file <none>
I20260619 20:35:43.020357 23041 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:43.020417 23041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:43.020540 23041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:43.021656 23041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "1598bf8c825343e382b77a7af7fae739"
format_stamp: "Formatted at 2026-06-19 20:35:43 on dist-test-slave-4m58"
server_key: "bd4af43e16eede18c1e5b614788a8c41"
server_key_iv: "16062a63e17fe9cd6db3cc1fc7190495"
server_key_version: "encryptionkey@0"
I20260619 20:35:43.021986 23041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "1598bf8c825343e382b77a7af7fae739"
format_stamp: "Formatted at 2026-06-19 20:35:43 on dist-test-slave-4m58"
server_key: "bd4af43e16eede18c1e5b614788a8c41"
server_key_iv: "16062a63e17fe9cd6db3cc1fc7190495"
server_key_version: "encryptionkey@0"
I20260619 20:35:43.023488 23041 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 20:35:43.024463 23055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:43.024672 23041 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:35:43.024767 23041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "1598bf8c825343e382b77a7af7fae739"
format_stamp: "Formatted at 2026-06-19 20:35:43 on dist-test-slave-4m58"
server_key: "bd4af43e16eede18c1e5b614788a8c41"
server_key_iv: "16062a63e17fe9cd6db3cc1fc7190495"
server_key_version: "encryptionkey@0"
I20260619 20:35:43.024860 23041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:43.034427 23041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:43.034785 23041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:43.034921 23041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:43.038655 23041 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:34423
I20260619 20:35:43.038690 23107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:34423 every 8 connection(s)
I20260619 20:35:43.039013 23041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:35:43.039564 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 23041
I20260619 20:35:43.039680 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:35:43.039932 4429 external_mini_cluster.cc:1428] Setting key 9760de143cc4f432ebcf9c3e52a0a66b
I20260619 20:35:43.040182 23108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:43.043428 23108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1598bf8c825343e382b77a7af7fae739: Bootstrap starting.
I20260619 20:35:43.044101 23108 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1598bf8c825343e382b77a7af7fae739: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:43.044390 23108 log.cc:824] T 00000000000000000000000000000000 P 1598bf8c825343e382b77a7af7fae739: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:43.045572 23108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1598bf8c825343e382b77a7af7fae739: No bootstrap required, opened a new log
I20260619 20:35:43.046183 23111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:43.040507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47928 (local address 127.4.83.126:34423)
0619 20:35:43.040713 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:43.040723 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:43.040755 (+ 32us) server_negotiation.cc:407] Connection header received
0619 20:35:43.041286 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:43.041293 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:43.041355 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:43.041469 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:43.041712 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.042574 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.043623 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.044129 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.044265 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:43.045009 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:43.045013 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:43.045014 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:35:43.045668 (+ 654us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:43.045716 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:35:43.045735 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:43.045801 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:43.045888 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 20:35:43.045971 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
I20260619 20:35:43.047560 23108 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1598bf8c825343e382b77a7af7fae739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1598bf8c825343e382b77a7af7fae739" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 34423 } }
I20260619 20:35:43.047698 23108 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1598bf8c825343e382b77a7af7fae739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:43.047720 23108 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1598bf8c825343e382b77a7af7fae739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1598bf8c825343e382b77a7af7fae739, State: Initialized, Role: FOLLOWER
I20260619 20:35:43.047853 23108 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1598bf8c825343e382b77a7af7fae739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1598bf8c825343e382b77a7af7fae739" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 34423 } }
I20260619 20:35:43.047937 23108 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1598bf8c825343e382b77a7af7fae739 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:35:43.047971 23108 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1598bf8c825343e382b77a7af7fae739 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:35:43.048017 23108 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1598bf8c825343e382b77a7af7fae739 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:43.048616 23108 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1598bf8c825343e382b77a7af7fae739 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1598bf8c825343e382b77a7af7fae739" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 34423 } }
I20260619 20:35:43.048708 23108 leader_election.cc:302] T 00000000000000000000000000000000 P 1598bf8c825343e382b77a7af7fae739 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1598bf8c825343e382b77a7af7fae739; no voters:
I20260619 20:35:43.048899 23108 leader_election.cc:288] T 00000000000000000000000000000000 P 1598bf8c825343e382b77a7af7fae739 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:35:43.049013 23113 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1598bf8c825343e382b77a7af7fae739 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:35:43.049170 23113 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1598bf8c825343e382b77a7af7fae739 [term 1 LEADER]: Becoming Leader. State: Replica: 1598bf8c825343e382b77a7af7fae739, State: Running, Role: LEADER
I20260619 20:35:43.049190 23108 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1598bf8c825343e382b77a7af7fae739 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:35:43.049261 23113 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1598bf8c825343e382b77a7af7fae739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1598bf8c825343e382b77a7af7fae739" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 34423 } }
I20260619 20:35:43.049666 23114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1598bf8c825343e382b77a7af7fae739 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1598bf8c825343e382b77a7af7fae739" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1598bf8c825343e382b77a7af7fae739" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 34423 } } }
I20260619 20:35:43.049747 23114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1598bf8c825343e382b77a7af7fae739 [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:43.049902 23115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1598bf8c825343e382b77a7af7fae739 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1598bf8c825343e382b77a7af7fae739. Latest consensus state: current_term: 1 leader_uuid: "1598bf8c825343e382b77a7af7fae739" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1598bf8c825343e382b77a7af7fae739" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 34423 } } }
I20260619 20:35:43.050040 23115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1598bf8c825343e382b77a7af7fae739 [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:43.050254 23117 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:35:43.050771 23117 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:35:43.053346 23117 catalog_manager.cc:1347] Generated new cluster ID: 57a56c15e2834ced8a29c08341725dea
I20260619 20:35:43.053408 23117 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:35:43.078014 23117 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:35:43.078603 23117 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:35:43.081220 23117 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1598bf8c825343e382b77a7af7fae739: Generated new TSK 0
I20260619 20:35:43.081482 23117 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:35:43.083990 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:0
--local_ip_for_outbound_sockets=127.4.83.65
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:34423
--builtin_ntp_servers=127.4.83.84:44009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 20:35:43.189875 23132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:43.190119 23132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:43.190155 23132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:43.191450 23132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:43.191514 23132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:43.191555 23132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:35:43.193539 23132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:44009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:34423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:43.193982 23132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:43.194412 23132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:43.201295 23138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:43.201295 23137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:43.201318 23140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:43.201843 23132 server_base.cc:1034] running on GCE node
I20260619 20:35:43.202090 23132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:43.202395 23132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:43.203568 23132 hybrid_clock.cc:648] HybridClock initialized: now 1781901343203522 us; error 64 us; skew 500 ppm
I20260619 20:35:43.205226 23132 webserver.cc:460] Webserver started at http://127.4.83.65:40925/ using document root <none> and password file <none>
I20260619 20:35:43.205509 23132 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:43.205569 23132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:43.205694 23132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:43.207175 23132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-0/data/instance:
uuid: "7986b5f53bcf409286a18828685fccc4"
format_stamp: "Formatted at 2026-06-19 20:35:43 on dist-test-slave-4m58"
server_key: "9ede66e254c50923dd64f552aca0c90a"
server_key_iv: "8079ae9a96253e49eee546335615d69b"
server_key_version: "encryptionkey@0"
I20260619 20:35:43.207571 23132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance:
uuid: "7986b5f53bcf409286a18828685fccc4"
format_stamp: "Formatted at 2026-06-19 20:35:43 on dist-test-slave-4m58"
server_key: "9ede66e254c50923dd64f552aca0c90a"
server_key_iv: "8079ae9a96253e49eee546335615d69b"
server_key_version: "encryptionkey@0"
I20260619 20:35:43.209436 23132 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260619 20:35:43.210276 23146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:43.210441 23132 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:35:43.210527 23132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "7986b5f53bcf409286a18828685fccc4"
format_stamp: "Formatted at 2026-06-19 20:35:43 on dist-test-slave-4m58"
server_key: "9ede66e254c50923dd64f552aca0c90a"
server_key_iv: "8079ae9a96253e49eee546335615d69b"
server_key_version: "encryptionkey@0"
I20260619 20:35:43.210608 23132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:43.234426 23132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:43.234743 23132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:43.234864 23132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:43.235101 23132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:35:43.235410 23132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:35:43.235442 23132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:43.235466 23132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:35:43.235476 23132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:43.241159 23132 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:34523
I20260619 20:35:43.241179 23259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:34523 every 8 connection(s)
I20260619 20:35:43.241562 23132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
I20260619 20:35:43.246657 23111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:43.242279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:39739 (local address 127.4.83.126:34423)
0619 20:35:43.242383 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:43.242385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:43.242586 (+ 201us) server_negotiation.cc:407] Connection header received
0619 20:35:43.243051 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:43.243052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:43.243063 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:43.243128 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:43.243816 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.244345 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.245361 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.245575 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.245628 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:43.246406 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:43.246407 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:43.246407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:35:43.246412 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:43.246445 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:35:43.246449 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:43.246493 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:43.246557 (+ 64us) server_negotiation.cc:299] Negotiation successful
0619 20:35:43.246593 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 20:35:43.246812 23262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:43.242240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:34423 (local address 127.4.83.65:39739)
0619 20:35:43.242432 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:43.242482 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:43.242964 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:43.243188 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:43.243191 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:43.243198 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:35:43.243708 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:43.243713 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.244434 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.244435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:43.245276 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:43.245280 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.245648 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.245649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:43.245765 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:43.246269 (+ 504us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:35:43.246269 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:35:43.246299 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:35:43.246322 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:35:43.246498 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:35:43.246501 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:35:43.246503 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:35:43.246549 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 20:35:43.246628 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":85,"threads_started":1}
I20260619 20:35:43.247304 23260 heartbeater.cc:347] Connected to a master server at 127.4.83.126:34423
I20260619 20:35:43.247401 23260 heartbeater.cc:464] Registering TS with master...
I20260619 20:35:43.247604 23260 heartbeater.cc:511] Master 127.4.83.126:34423 requested a full tablet report, sending...
I20260619 20:35:43.248088 23072 ts_manager.cc:194] Registered new tserver with Master: 7986b5f53bcf409286a18828685fccc4 (127.4.83.65:34523)
I20260619 20:35:43.248706 23072 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.83.65:39739
I20260619 20:35:43.250298 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 23132
I20260619 20:35:43.250382 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance
I20260619 20:35:43.250590 4429 external_mini_cluster.cc:1428] Setting key b4f44cc87eef2309f74edf78868ae320
I20260619 20:35:43.251451 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:0
--local_ip_for_outbound_sockets=127.4.83.66
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:34423
--builtin_ntp_servers=127.4.83.84:44009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 20:35:43.331964 23263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:43.332185 23263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:43.332214 23263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:43.333396 23263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:43.333443 23263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:43.333478 23263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:35:43.334849 23263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:44009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:34423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:43.335108 23263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:43.335398 23263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:43.340699 23268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:43.340718 23271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:43.340699 23269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:43.341475 23263 server_base.cc:1034] running on GCE node
I20260619 20:35:43.341681 23263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:43.341960 23263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:43.343118 23263 hybrid_clock.cc:648] HybridClock initialized: now 1781901343343108 us; error 31 us; skew 500 ppm
I20260619 20:35:43.344417 23263 webserver.cc:460] Webserver started at http://127.4.83.66:37323/ using document root <none> and password file <none>
I20260619 20:35:43.344663 23263 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:43.344712 23263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:43.344836 23263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:43.345923 23263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-1/data/instance:
uuid: "eac570be08f24dbeb96f3dab6409a679"
format_stamp: "Formatted at 2026-06-19 20:35:43 on dist-test-slave-4m58"
server_key: "b6c8c94cedb03eaeb12dde7f2d7805b4"
server_key_iv: "d513b55eb8e29a9151ea5e9d0266bfe4"
server_key_version: "encryptionkey@0"
I20260619 20:35:43.346251 23263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance:
uuid: "eac570be08f24dbeb96f3dab6409a679"
format_stamp: "Formatted at 2026-06-19 20:35:43 on dist-test-slave-4m58"
server_key: "b6c8c94cedb03eaeb12dde7f2d7805b4"
server_key_iv: "d513b55eb8e29a9151ea5e9d0266bfe4"
server_key_version: "encryptionkey@0"
I20260619 20:35:43.347519 23263 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:35:43.348511 23277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:43.348716 23263 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:35:43.348806 23263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "eac570be08f24dbeb96f3dab6409a679"
format_stamp: "Formatted at 2026-06-19 20:35:43 on dist-test-slave-4m58"
server_key: "b6c8c94cedb03eaeb12dde7f2d7805b4"
server_key_iv: "d513b55eb8e29a9151ea5e9d0266bfe4"
server_key_version: "encryptionkey@0"
I20260619 20:35:43.348881 23263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:43.359879 23263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:43.360220 23263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:43.360351 23263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:43.360584 23263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:35:43.360905 23263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:35:43.360940 23263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:43.360971 23263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:35:43.360996 23263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:43.367128 23263 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:37091
I20260619 20:35:43.367168 23390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:37091 every 8 connection(s)
I20260619 20:35:43.367501 23263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
I20260619 20:35:43.372660 23111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:43.368253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:39701 (local address 127.4.83.126:34423)
0619 20:35:43.368334 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:43.368336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:43.368530 (+ 194us) server_negotiation.cc:407] Connection header received
0619 20:35:43.369056 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:43.369058 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:43.369070 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:43.369137 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:43.369807 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.370302 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.371248 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.371439 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.371501 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:43.372391 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:43.372393 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:43.372393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:35:43.372399 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:43.372413 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:35:43.372417 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:43.372460 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:43.372539 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 20:35:43.372571 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 20:35:43.372876 23393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:43.368219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:34423 (local address 127.4.83.66:39701)
0619 20:35:43.368410 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:43.368447 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:43.368957 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:43.369211 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:43.369214 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:43.369221 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:35:43.369718 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:43.369723 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.370386 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.370387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:43.371166 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:43.371169 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.371516 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.371517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:43.371642 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:43.372207 (+ 565us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:35:43.372207 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:35:43.372253 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:35:43.372277 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:35:43.372469 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:35:43.372474 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:35:43.372478 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:35:43.372544 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 20:35:43.372656 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":68,"threads_started":1}
I20260619 20:35:43.373709 23391 heartbeater.cc:347] Connected to a master server at 127.4.83.126:34423
I20260619 20:35:43.373811 23391 heartbeater.cc:464] Registering TS with master...
I20260619 20:35:43.374018 23391 heartbeater.cc:511] Master 127.4.83.126:34423 requested a full tablet report, sending...
I20260619 20:35:43.374644 23072 ts_manager.cc:194] Registered new tserver with Master: eac570be08f24dbeb96f3dab6409a679 (127.4.83.66:37091)
I20260619 20:35:43.375047 23072 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.83.66:39701
I20260619 20:35:43.376264 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 23263
I20260619 20:35:43.376346 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance
I20260619 20:35:43.376530 4429 external_mini_cluster.cc:1428] Setting key 9ce2e366c79a14849b07f45507522f9e
I20260619 20:35:43.377501 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:0
--local_ip_for_outbound_sockets=127.4.83.67
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:34423
--builtin_ntp_servers=127.4.83.84:44009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 20:35:43.460808 23394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:43.461040 23394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:43.461072 23394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:43.462352 23394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:43.462401 23394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:43.462440 23394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:35:43.464010 23394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:44009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:34423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:43.464303 23394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:43.464624 23394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:43.469960 23400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:43.469991 23394 server_base.cc:1034] running on GCE node
W20260619 20:35:43.469969 23399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:43.469950 23402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:43.470396 23394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:43.470650 23394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:43.471848 23394 hybrid_clock.cc:648] HybridClock initialized: now 1781901343471809 us; error 30 us; skew 500 ppm
I20260619 20:35:43.473189 23394 webserver.cc:460] Webserver started at http://127.4.83.67:40053/ using document root <none> and password file <none>
I20260619 20:35:43.473479 23394 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:43.473534 23394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:43.473667 23394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:43.474757 23394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-2/data/instance:
uuid: "52ada1ca63b344c09ef1a86316cc30b9"
format_stamp: "Formatted at 2026-06-19 20:35:43 on dist-test-slave-4m58"
server_key: "2ca761ca344fd6d467b678867956f2d2"
server_key_iv: "21931d9973a3236971a6b0b9669909e2"
server_key_version: "encryptionkey@0"
I20260619 20:35:43.475116 23394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance:
uuid: "52ada1ca63b344c09ef1a86316cc30b9"
format_stamp: "Formatted at 2026-06-19 20:35:43 on dist-test-slave-4m58"
server_key: "2ca761ca344fd6d467b678867956f2d2"
server_key_iv: "21931d9973a3236971a6b0b9669909e2"
server_key_version: "encryptionkey@0"
I20260619 20:35:43.476544 23394 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:35:43.477418 23408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:43.477605 23394 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:43.477690 23394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "52ada1ca63b344c09ef1a86316cc30b9"
format_stamp: "Formatted at 2026-06-19 20:35:43 on dist-test-slave-4m58"
server_key: "2ca761ca344fd6d467b678867956f2d2"
server_key_iv: "21931d9973a3236971a6b0b9669909e2"
server_key_version: "encryptionkey@0"
I20260619 20:35:43.477753 23394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:43.493862 23394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:43.494203 23394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:43.494347 23394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:43.494586 23394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:35:43.494907 23394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:35:43.494941 23394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:43.494971 23394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:35:43.494987 23394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:43.500584 23394 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:45655
I20260619 20:35:43.500626 23521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:45655 every 8 connection(s)
I20260619 20:35:43.500945 23394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
I20260619 20:35:43.502452 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 23394
I20260619 20:35:43.502574 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance
I20260619 20:35:43.502748 4429 external_mini_cluster.cc:1428] Setting key 068d4be01e65fcfe4d9c52ac537cd8f8
I20260619 20:35:43.506400 23111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:43.501665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:52297 (local address 127.4.83.126:34423)
0619 20:35:43.501727 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:43.501729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:43.501938 (+ 209us) server_negotiation.cc:407] Connection header received
0619 20:35:43.502468 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:43.502470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:43.502483 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:43.502534 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:43.503493 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.504093 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.505101 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.505288 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.505345 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:43.506174 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:43.506175 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:43.506175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:35:43.506180 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:43.506196 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:35:43.506200 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:43.506258 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:43.506314 (+ 56us) server_negotiation.cc:299] Negotiation successful
0619 20:35:43.506338 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 20:35:43.506637 23524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:43.501636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:34423 (local address 127.4.83.67:52297)
0619 20:35:43.501822 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:43.501857 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:43.502365 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:43.502603 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:43.502607 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:43.502617 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:35:43.503420 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:43.503428 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.504188 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.504189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:43.505025 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:43.505028 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.505395 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.505396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:43.505518 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:43.506017 (+ 499us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:35:43.506018 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:35:43.506047 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:35:43.506072 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:35:43.506261 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:35:43.506264 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:35:43.506267 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:35:43.506314 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 20:35:43.506406 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":86,"threads_started":1}
I20260619 20:35:43.507164 23522 heartbeater.cc:347] Connected to a master server at 127.4.83.126:34423
I20260619 20:35:43.507256 23522 heartbeater.cc:464] Registering TS with master...
I20260619 20:35:43.507442 23522 heartbeater.cc:511] Master 127.4.83.126:34423 requested a full tablet report, sending...
I20260619 20:35:43.507788 23072 ts_manager.cc:194] Registered new tserver with Master: 52ada1ca63b344c09ef1a86316cc30b9 (127.4.83.67:45655)
I20260619 20:35:43.508196 23072 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.83.67:52297
I20260619 20:35:43.514560 4429 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:35:43.518959 23111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:43.515979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47940 (local address 127.4.83.126:34423)
0619 20:35:43.516032 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:43.516034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:43.516176 (+ 142us) server_negotiation.cc:407] Connection header received
0619 20:35:43.516205 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:43.516206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:43.516216 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:43.516273 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:43.516431 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.516925 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.517707 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.517906 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.517969 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:43.518658 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:43.518659 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:43.518660 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:35:43.518665 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:43.518680 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:35:43.518684 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:43.518752 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:43.518829 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 20:35:43.518840 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 20:35:43.520809 23072 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47940:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:35:43.522742 23072 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:35:43.529057 23533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:43.525087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49010 (local address 127.4.83.67:45655)
0619 20:35:43.525272 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:43.525277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:43.525288 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:35:43.525517 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:43.525519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:43.525531 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:43.525591 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:43.525789 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.527053 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.528061 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.528651 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.528719 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:43.528817 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:43.528921 (+ 104us) server_negotiation.cc:299] Negotiation successful
0619 20:35:43.528959 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":82,"threads_started":1}
I20260619 20:35:43.529045 23532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:43.524985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:45655 (local address 127.0.0.1:49010)
0619 20:35:43.525174 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:43.525216 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:43.525283 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:43.525659 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:43.525660 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:43.525661 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:43.525689 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:43.525690 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.527112 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.527113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:43.527967 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:43.527970 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.528729 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.528730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:43.528812 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:43.528825 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 20:35:43.528905 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 20:35:43.528960 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":90,"threads_started":1}
I20260619 20:35:43.529438 23531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:43.524987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:34523 (local address 127.0.0.1:39082)
0619 20:35:43.525335 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:43.525344 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:43.525386 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:43.525591 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:43.525593 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:43.525596 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:43.525676 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:43.525679 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.527044 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.527045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:43.528152 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:43.528154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.529186 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.529187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:43.529264 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:43.529267 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:35:43.529324 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 20:35:43.529365 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":56,"threads_started":1}
I20260619 20:35:43.529479 23534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:43.525233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39082 (local address 127.4.83.65:34523)
0619 20:35:43.525417 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:43.525422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:43.525433 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:35:43.525474 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:43.525476 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:43.525492 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:43.525580 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:43.525789 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.526957 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.528238 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.528997 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.529065 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:43.529147 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:43.529352 (+ 205us) server_negotiation.cc:299] Negotiation successful
0619 20:35:43.529400 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":81,"threads_started":1}
I20260619 20:35:43.529839 23535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:43.525233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38652 (local address 127.4.83.66:37091)
0619 20:35:43.525448 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:43.525451 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:43.525459 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:35:43.525492 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:43.525494 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:43.525506 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:43.525580 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:43.525903 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.526731 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.528312 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.529009 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.529065 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:43.529289 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:43.529712 (+ 423us) server_negotiation.cc:299] Negotiation successful
0619 20:35:43.529758 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":113,"threads_started":1}
I20260619 20:35:43.530095 23530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:43.524987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:37091 (local address 127.0.0.1:38652)
0619 20:35:43.525389 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:43.525397 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:43.525429 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:43.525757 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:43.525757 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:43.525759 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:43.525785 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:43.525786 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.526812 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.526813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:43.528237 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:43.528239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.529330 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.529331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:43.529382 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:43.529385 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:35:43.529428 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 20:35:43.529460 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":1,"thread_start_us":68,"threads_started":1}
I20260619 20:35:43.530054 23456 tablet_service.cc:1467] Processing CreateTablet for tablet b146262192884bdf8b3eef49e8a645b4 (DEFAULT_TABLE table=test-table [id=93643fbddec64c7382e48478dcadad91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:35:43.530134 23194 tablet_service.cc:1467] Processing CreateTablet for tablet b146262192884bdf8b3eef49e8a645b4 (DEFAULT_TABLE table=test-table [id=93643fbddec64c7382e48478dcadad91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:35:43.530472 23456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b146262192884bdf8b3eef49e8a645b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:43.530488 23194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b146262192884bdf8b3eef49e8a645b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:43.533186 23537 tablet_bootstrap.cc:492] T b146262192884bdf8b3eef49e8a645b4 P 7986b5f53bcf409286a18828685fccc4: Bootstrap starting.
I20260619 20:35:43.533808 23325 tablet_service.cc:1467] Processing CreateTablet for tablet b146262192884bdf8b3eef49e8a645b4 (DEFAULT_TABLE table=test-table [id=93643fbddec64c7382e48478dcadad91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:35:43.534193 23537 tablet_bootstrap.cc:654] T b146262192884bdf8b3eef49e8a645b4 P 7986b5f53bcf409286a18828685fccc4: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:43.534317 23325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b146262192884bdf8b3eef49e8a645b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:43.534572 23537 log.cc:824] T b146262192884bdf8b3eef49e8a645b4 P 7986b5f53bcf409286a18828685fccc4: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:43.534988 23536 tablet_bootstrap.cc:492] T b146262192884bdf8b3eef49e8a645b4 P 52ada1ca63b344c09ef1a86316cc30b9: Bootstrap starting.
I20260619 20:35:43.535670 23536 tablet_bootstrap.cc:654] T b146262192884bdf8b3eef49e8a645b4 P 52ada1ca63b344c09ef1a86316cc30b9: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:43.536031 23536 log.cc:824] T b146262192884bdf8b3eef49e8a645b4 P 52ada1ca63b344c09ef1a86316cc30b9: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:43.536650 23537 tablet_bootstrap.cc:492] T b146262192884bdf8b3eef49e8a645b4 P 7986b5f53bcf409286a18828685fccc4: No bootstrap required, opened a new log
I20260619 20:35:43.536728 23537 ts_tablet_manager.cc:1397] T b146262192884bdf8b3eef49e8a645b4 P 7986b5f53bcf409286a18828685fccc4: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 20:35:43.537038 23536 tablet_bootstrap.cc:492] T b146262192884bdf8b3eef49e8a645b4 P 52ada1ca63b344c09ef1a86316cc30b9: No bootstrap required, opened a new log
I20260619 20:35:43.537112 23536 ts_tablet_manager.cc:1397] T b146262192884bdf8b3eef49e8a645b4 P 52ada1ca63b344c09ef1a86316cc30b9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 20:35:43.538112 23540 tablet_bootstrap.cc:492] T b146262192884bdf8b3eef49e8a645b4 P eac570be08f24dbeb96f3dab6409a679: Bootstrap starting.
I20260619 20:35:43.538930 23540 tablet_bootstrap.cc:654] T b146262192884bdf8b3eef49e8a645b4 P eac570be08f24dbeb96f3dab6409a679: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:43.539258 23540 log.cc:824] T b146262192884bdf8b3eef49e8a645b4 P eac570be08f24dbeb96f3dab6409a679: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:43.539722 23536 raft_consensus.cc:348] T b146262192884bdf8b3eef49e8a645b4 P 52ada1ca63b344c09ef1a86316cc30b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986b5f53bcf409286a18828685fccc4" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34523 } } peers { permanent_uuid: "eac570be08f24dbeb96f3dab6409a679" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37091 } } peers { permanent_uuid: "52ada1ca63b344c09ef1a86316cc30b9" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 45655 } }
I20260619 20:35:43.539925 23536 raft_consensus.cc:374] T b146262192884bdf8b3eef49e8a645b4 P 52ada1ca63b344c09ef1a86316cc30b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:43.539798 23537 raft_consensus.cc:348] T b146262192884bdf8b3eef49e8a645b4 P 7986b5f53bcf409286a18828685fccc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986b5f53bcf409286a18828685fccc4" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34523 } } peers { permanent_uuid: "eac570be08f24dbeb96f3dab6409a679" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37091 } } peers { permanent_uuid: "52ada1ca63b344c09ef1a86316cc30b9" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 45655 } }
I20260619 20:35:43.539973 23536 raft_consensus.cc:729] T b146262192884bdf8b3eef49e8a645b4 P 52ada1ca63b344c09ef1a86316cc30b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ada1ca63b344c09ef1a86316cc30b9, State: Initialized, Role: FOLLOWER
I20260619 20:35:43.539997 23537 raft_consensus.cc:374] T b146262192884bdf8b3eef49e8a645b4 P 7986b5f53bcf409286a18828685fccc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:43.540028 23537 raft_consensus.cc:729] T b146262192884bdf8b3eef49e8a645b4 P 7986b5f53bcf409286a18828685fccc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7986b5f53bcf409286a18828685fccc4, State: Initialized, Role: FOLLOWER
I20260619 20:35:43.540095 23536 consensus_queue.cc:260] T b146262192884bdf8b3eef49e8a645b4 P 52ada1ca63b344c09ef1a86316cc30b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986b5f53bcf409286a18828685fccc4" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34523 } } peers { permanent_uuid: "eac570be08f24dbeb96f3dab6409a679" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37091 } } peers { permanent_uuid: "52ada1ca63b344c09ef1a86316cc30b9" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 45655 } }
I20260619 20:35:43.540189 23540 tablet_bootstrap.cc:492] T b146262192884bdf8b3eef49e8a645b4 P eac570be08f24dbeb96f3dab6409a679: No bootstrap required, opened a new log
I20260619 20:35:43.540182 23537 consensus_queue.cc:260] T b146262192884bdf8b3eef49e8a645b4 P 7986b5f53bcf409286a18828685fccc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986b5f53bcf409286a18828685fccc4" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34523 } } peers { permanent_uuid: "eac570be08f24dbeb96f3dab6409a679" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37091 } } peers { permanent_uuid: "52ada1ca63b344c09ef1a86316cc30b9" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 45655 } }
I20260619 20:35:43.540268 23540 ts_tablet_manager.cc:1397] T b146262192884bdf8b3eef49e8a645b4 P eac570be08f24dbeb96f3dab6409a679: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260619 20:35:43.540371 23536 ts_tablet_manager.cc:1428] T b146262192884bdf8b3eef49e8a645b4 P 52ada1ca63b344c09ef1a86316cc30b9: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260619 20:35:43.540468 23537 ts_tablet_manager.cc:1428] T b146262192884bdf8b3eef49e8a645b4 P 7986b5f53bcf409286a18828685fccc4: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260619 20:35:43.540503 23522 heartbeater.cc:503] Master 127.4.83.126:34423 was elected leader, sending a full tablet report...
I20260619 20:35:43.540548 23260 heartbeater.cc:503] Master 127.4.83.126:34423 was elected leader, sending a full tablet report...
I20260619 20:35:43.542748 23540 raft_consensus.cc:348] T b146262192884bdf8b3eef49e8a645b4 P eac570be08f24dbeb96f3dab6409a679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986b5f53bcf409286a18828685fccc4" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34523 } } peers { permanent_uuid: "eac570be08f24dbeb96f3dab6409a679" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37091 } } peers { permanent_uuid: "52ada1ca63b344c09ef1a86316cc30b9" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 45655 } }
I20260619 20:35:43.542892 23540 raft_consensus.cc:374] T b146262192884bdf8b3eef49e8a645b4 P eac570be08f24dbeb96f3dab6409a679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:43.542914 23540 raft_consensus.cc:729] T b146262192884bdf8b3eef49e8a645b4 P eac570be08f24dbeb96f3dab6409a679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac570be08f24dbeb96f3dab6409a679, State: Initialized, Role: FOLLOWER
I20260619 20:35:43.543008 23540 consensus_queue.cc:260] T b146262192884bdf8b3eef49e8a645b4 P eac570be08f24dbeb96f3dab6409a679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986b5f53bcf409286a18828685fccc4" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34523 } } peers { permanent_uuid: "eac570be08f24dbeb96f3dab6409a679" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37091 } } peers { permanent_uuid: "52ada1ca63b344c09ef1a86316cc30b9" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 45655 } }
I20260619 20:35:43.543222 23540 ts_tablet_manager.cc:1428] T b146262192884bdf8b3eef49e8a645b4 P eac570be08f24dbeb96f3dab6409a679: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 20:35:43.543277 23391 heartbeater.cc:503] Master 127.4.83.126:34423 was elected leader, sending a full tablet report...
I20260619 20:35:43.570561 23543 raft_consensus.cc:482] T b146262192884bdf8b3eef49e8a645b4 P 7986b5f53bcf409286a18828685fccc4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:35:43.570690 23543 raft_consensus.cc:504] T b146262192884bdf8b3eef49e8a645b4 P 7986b5f53bcf409286a18828685fccc4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986b5f53bcf409286a18828685fccc4" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34523 } } peers { permanent_uuid: "eac570be08f24dbeb96f3dab6409a679" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37091 } } peers { permanent_uuid: "52ada1ca63b344c09ef1a86316cc30b9" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 45655 } }
I20260619 20:35:43.571053 23543 leader_election.cc:288] T b146262192884bdf8b3eef49e8a645b4 P 7986b5f53bcf409286a18828685fccc4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac570be08f24dbeb96f3dab6409a679 (127.4.83.66:37091), 52ada1ca63b344c09ef1a86316cc30b9 (127.4.83.67:45655)
I20260619 20:35:43.574833 23533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:43.571325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:43283 (local address 127.4.83.67:45655)
0619 20:35:43.571372 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:43.571374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:43.571385 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:35:43.571416 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:43.571416 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:43.571425 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:43.571487 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:43.571663 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.572351 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.573832 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.574429 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.574494 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:43.574506 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:43.574711 (+ 205us) server_negotiation.cc:299] Negotiation successful
0619 20:35:43.574756 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:35:43.574831 23535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:43.571322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:43693 (local address 127.4.83.66:37091)
0619 20:35:43.571379 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:43.571380 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:43.571487 (+ 107us) server_negotiation.cc:407] Connection header received
0619 20:35:43.571585 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:43.571586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:43.571596 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:43.571646 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:43.571784 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.572349 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.573831 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.574402 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.574512 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:43.574522 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:43.574718 (+ 196us) server_negotiation.cc:299] Negotiation successful
0619 20:35:43.574760 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 20:35:43.574867 23545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:43.571257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:37091 (local address 127.4.83.65:43693)
0619 20:35:43.571436 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:43.571443 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:43.571493 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:43.571672 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:43.571673 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:43.571674 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:43.571728 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:43.571730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.572456 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.572457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:43.573718 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:43.573719 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.574525 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.574526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:43.574635 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:43.574638 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:35:43.574692 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 20:35:43.574747 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":9,"thread_start_us":96,"threads_started":1}
I20260619 20:35:43.574884 23262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:43.571236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:45655 (local address 127.4.83.65:43283)
0619 20:35:43.571293 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:43.571304 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:43.571348 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:43.571508 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:43.571508 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:43.571510 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:43.571592 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:43.571594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.572434 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.572435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:43.573714 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:43.573717 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.574525 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.574526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:43.574627 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:43.574632 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:35:43.574698 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 20:35:43.574752 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":86}
I20260619 20:35:43.575279 23345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b146262192884bdf8b3eef49e8a645b4" candidate_uuid: "7986b5f53bcf409286a18828685fccc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac570be08f24dbeb96f3dab6409a679" is_pre_election: true
I20260619 20:35:43.575454 23345 raft_consensus.cc:2454] T b146262192884bdf8b3eef49e8a645b4 P eac570be08f24dbeb96f3dab6409a679 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7986b5f53bcf409286a18828685fccc4 in term 0.
I20260619 20:35:43.575484 23476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b146262192884bdf8b3eef49e8a645b4" candidate_uuid: "7986b5f53bcf409286a18828685fccc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ada1ca63b344c09ef1a86316cc30b9" is_pre_election: true
I20260619 20:35:43.575657 23476 raft_consensus.cc:2454] T b146262192884bdf8b3eef49e8a645b4 P 52ada1ca63b344c09ef1a86316cc30b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7986b5f53bcf409286a18828685fccc4 in term 0.
I20260619 20:35:43.575681 23149 leader_election.cc:302] T b146262192884bdf8b3eef49e8a645b4 P 7986b5f53bcf409286a18828685fccc4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7986b5f53bcf409286a18828685fccc4, eac570be08f24dbeb96f3dab6409a679; no voters:
I20260619 20:35:43.575811 23543 raft_consensus.cc:2781] T b146262192884bdf8b3eef49e8a645b4 P 7986b5f53bcf409286a18828685fccc4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:35:43.575908 23543 raft_consensus.cc:482] T b146262192884bdf8b3eef49e8a645b4 P 7986b5f53bcf409286a18828685fccc4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:35:43.575941 23543 raft_consensus.cc:3037] T b146262192884bdf8b3eef49e8a645b4 P 7986b5f53bcf409286a18828685fccc4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:43.576658 23543 raft_consensus.cc:504] T b146262192884bdf8b3eef49e8a645b4 P 7986b5f53bcf409286a18828685fccc4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986b5f53bcf409286a18828685fccc4" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34523 } } peers { permanent_uuid: "eac570be08f24dbeb96f3dab6409a679" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37091 } } peers { permanent_uuid: "52ada1ca63b344c09ef1a86316cc30b9" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 45655 } }
I20260619 20:35:43.576792 23543 leader_election.cc:288] T b146262192884bdf8b3eef49e8a645b4 P 7986b5f53bcf409286a18828685fccc4 [CANDIDATE]: Term 1 election: Requested vote from peers eac570be08f24dbeb96f3dab6409a679 (127.4.83.66:37091), 52ada1ca63b344c09ef1a86316cc30b9 (127.4.83.67:45655)
I20260619 20:35:43.576954 23345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b146262192884bdf8b3eef49e8a645b4" candidate_uuid: "7986b5f53bcf409286a18828685fccc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac570be08f24dbeb96f3dab6409a679"
I20260619 20:35:43.576948 23476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b146262192884bdf8b3eef49e8a645b4" candidate_uuid: "7986b5f53bcf409286a18828685fccc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ada1ca63b344c09ef1a86316cc30b9"
I20260619 20:35:43.577040 23476 raft_consensus.cc:3037] T b146262192884bdf8b3eef49e8a645b4 P 52ada1ca63b344c09ef1a86316cc30b9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:43.577040 23345 raft_consensus.cc:3037] T b146262192884bdf8b3eef49e8a645b4 P eac570be08f24dbeb96f3dab6409a679 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:43.577950 23476 raft_consensus.cc:2454] T b146262192884bdf8b3eef49e8a645b4 P 52ada1ca63b344c09ef1a86316cc30b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7986b5f53bcf409286a18828685fccc4 in term 1.
I20260619 20:35:43.577950 23345 raft_consensus.cc:2454] T b146262192884bdf8b3eef49e8a645b4 P eac570be08f24dbeb96f3dab6409a679 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7986b5f53bcf409286a18828685fccc4 in term 1.
I20260619 20:35:43.578126 23148 leader_election.cc:302] T b146262192884bdf8b3eef49e8a645b4 P 7986b5f53bcf409286a18828685fccc4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ada1ca63b344c09ef1a86316cc30b9, 7986b5f53bcf409286a18828685fccc4; no voters:
I20260619 20:35:43.578236 23543 raft_consensus.cc:2781] T b146262192884bdf8b3eef49e8a645b4 P 7986b5f53bcf409286a18828685fccc4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:35:43.578382 23543 raft_consensus.cc:686] T b146262192884bdf8b3eef49e8a645b4 P 7986b5f53bcf409286a18828685fccc4 [term 1 LEADER]: Becoming Leader. State: Replica: 7986b5f53bcf409286a18828685fccc4, State: Running, Role: LEADER
I20260619 20:35:43.578464 23543 consensus_queue.cc:237] T b146262192884bdf8b3eef49e8a645b4 P 7986b5f53bcf409286a18828685fccc4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986b5f53bcf409286a18828685fccc4" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34523 } } peers { permanent_uuid: "eac570be08f24dbeb96f3dab6409a679" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37091 } } peers { permanent_uuid: "52ada1ca63b344c09ef1a86316cc30b9" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 45655 } }
I20260619 20:35:43.579308 23071 catalog_manager.cc:5495] T b146262192884bdf8b3eef49e8a645b4 P 7986b5f53bcf409286a18828685fccc4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7986b5f53bcf409286a18828685fccc4 (127.4.83.65). New cstate: current_term: 1 leader_uuid: "7986b5f53bcf409286a18828685fccc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7986b5f53bcf409286a18828685fccc4" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eac570be08f24dbeb96f3dab6409a679" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ada1ca63b344c09ef1a86316cc30b9" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 45655 } health_report { overall_health: UNKNOWN } } }
I20260619 20:35:43.589239 23534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:43.586964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39092 (local address 127.4.83.65:34523)
0619 20:35:43.587029 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:43.587031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:43.587038 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:35:43.587076 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:43.587077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:43.587097 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:43.587159 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:35:43.587275 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.587754 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.588514 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:43.588702 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:43.588755 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:43.588903 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:35:43.588994 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:35:43.589079 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:43.589131 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 20:35:43.589141 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 20:35:43.591238 23476 raft_consensus.cc:1261] T b146262192884bdf8b3eef49e8a645b4 P 52ada1ca63b344c09ef1a86316cc30b9 [term 1 FOLLOWER]: Refusing update from remote peer 7986b5f53bcf409286a18828685fccc4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:35:43.591262 23345 raft_consensus.cc:1261] T b146262192884bdf8b3eef49e8a645b4 P eac570be08f24dbeb96f3dab6409a679 [term 1 FOLLOWER]: Refusing update from remote peer 7986b5f53bcf409286a18828685fccc4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:35:43.591703 23543 consensus_queue.cc:1035] T b146262192884bdf8b3eef49e8a645b4 P 7986b5f53bcf409286a18828685fccc4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ada1ca63b344c09ef1a86316cc30b9" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 45655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:35:43.591914 23546 consensus_queue.cc:1035] T b146262192884bdf8b3eef49e8a645b4 P 7986b5f53bcf409286a18828685fccc4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac570be08f24dbeb96f3dab6409a679" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:35:43.594666 23550 mvcc.cc:205] Tried to move back new op lower bound from 7298667903346221056 to 7298667903297904640. Current Snapshot: MvccSnapshot[applied={T|T < 7298667903346221056}]
I20260619 20:35:43.595062 23551 mvcc.cc:205] Tried to move back new op lower bound from 7298667903346221056 to 7298667903297904640. Current Snapshot: MvccSnapshot[applied={T|T < 7298667903346221056}]
I20260619 20:35:43.597488 23072 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:35:43.597574 23072 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:35:43.598224 23072 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1598bf8c825343e382b77a7af7fae739: Sending DeleteTablet for 3 replicas of tablet b146262192884bdf8b3eef49e8a645b4
I20260619 20:35:43.598569 23194 tablet_service.cc:1514] Processing DeleteTablet for tablet b146262192884bdf8b3eef49e8a645b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:35:43 UTC) from {username='slave'} at 127.0.0.1:39082
I20260619 20:35:43.598596 23456 tablet_service.cc:1514] Processing DeleteTablet for tablet b146262192884bdf8b3eef49e8a645b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:35:43 UTC) from {username='slave'} at 127.0.0.1:49010
I20260619 20:35:43.598595 23325 tablet_service.cc:1514] Processing DeleteTablet for tablet b146262192884bdf8b3eef49e8a645b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:35:43 UTC) from {username='slave'} at 127.0.0.1:38652
I20260619 20:35:43.598805 23561 tablet_replica.cc:330] T b146262192884bdf8b3eef49e8a645b4 P 52ada1ca63b344c09ef1a86316cc30b9: stopping tablet replica
I20260619 20:35:43.598902 23561 raft_consensus.cc:2229] T b146262192884bdf8b3eef49e8a645b4 P 52ada1ca63b344c09ef1a86316cc30b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:35:43.599014 23561 raft_consensus.cc:2258] T b146262192884bdf8b3eef49e8a645b4 P 52ada1ca63b344c09ef1a86316cc30b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:35:43.599202 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 23132
I20260619 20:35:43.599225 23560 tablet_replica.cc:330] T b146262192884bdf8b3eef49e8a645b4 P eac570be08f24dbeb96f3dab6409a679: stopping tablet replica
I20260619 20:35:43.599901 23560 raft_consensus.cc:2229] T b146262192884bdf8b3eef49e8a645b4 P eac570be08f24dbeb96f3dab6409a679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:35:43.600025 23560 raft_consensus.cc:2258] T b146262192884bdf8b3eef49e8a645b4 P eac570be08f24dbeb96f3dab6409a679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:35:43.601475 23561 ts_tablet_manager.cc:1905] T b146262192884bdf8b3eef49e8a645b4 P 52ada1ca63b344c09ef1a86316cc30b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:35:43.603386 23561 ts_tablet_manager.cc:1918] T b146262192884bdf8b3eef49e8a645b4 P 52ada1ca63b344c09ef1a86316cc30b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:35:43.603477 23561 log.cc:1196] T b146262192884bdf8b3eef49e8a645b4 P 52ada1ca63b344c09ef1a86316cc30b9: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901102012824-4429-0/minicluster-data/ts-2/wal/wals/b146262192884bdf8b3eef49e8a645b4
I20260619 20:35:43.603711 23561 ts_tablet_manager.cc:1939] T b146262192884bdf8b3eef49e8a645b4 P 52ada1ca63b344c09ef1a86316cc30b9: Deleting consensus metadata
I20260619 20:35:43.604374 23057 catalog_manager.cc:4841] TS 52ada1ca63b344c09ef1a86316cc30b9 (127.4.83.67:45655): tablet b146262192884bdf8b3eef49e8a645b4 (table test-table [id=93643fbddec64c7382e48478dcadad91]) successfully deleted
W20260619 20:35:43.604923 23059 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.83.65:34523 (error 108)
I20260619 20:35:43.605108 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 23263
I20260619 20:35:43.605813 23530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:43.605688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:34523 (local address 127.0.0.1:39106)
0619 20:35:43.605743 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:43.605783 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.65:34523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 20:35:43.605958 23059 catalog_manager.cc:4584] TS 7986b5f53bcf409286a18828685fccc4 (127.4.83.65:34523): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b146262192884bdf8b3eef49e8a645b4: Network error: Client connection negotiation failed: client connection to 127.4.83.65:34523: connect: Connection refused (error 111)
I20260619 20:35:43.609694 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 23394
I20260619 20:35:43.609786 23530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:43.609661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:37091 (local address 127.0.0.1:38668)
0619 20:35:43.609713 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:43.609747 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.66:37091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 20:35:43.614326 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 23041
2026-06-19T20:35:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (713 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:35:43 dist-test-slave-4m58 krb5kdc[23565](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:35:43 dist-test-slave-4m58 krb5kdc[23565](info): set up 2 sockets
Jun 19 20:35:43 dist-test-slave-4m58 krb5kdc[23565](info): commencing operation
krb5kdc: starting...
W20260619 20:35:44.580529 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.948s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:35:44 dist-test-slave-4m58 krb5kdc[23565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:35:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:35:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
I20260619 20:35:44.699313 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:37835
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:37835
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:35:44.778424 23578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:44.778649 23578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:44.778672 23578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:44.779994 23578 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:35:44.780042 23578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:44.780062 23578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:44.780073 23578 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:35:44.780084 23578 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:35:44.781639 23578 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:37835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:37835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:44.781867 23578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:44.782059 23578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:44.786693 23583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:44.786692 23584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:44.786705 23586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:44.786911 23578 server_base.cc:1034] running on GCE node
I20260619 20:35:44.787155 23578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:44.787441 23578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:44.788592 23578 hybrid_clock.cc:648] HybridClock initialized: now 1781901344788574 us; error 37 us; skew 500 ppm
Jun 19 20:35:44 dist-test-slave-4m58 krb5kdc[23565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901344, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:35:44.791061 23578 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:35:44.791491 23578 webserver.cc:460] Webserver started at http://127.4.83.126:36957/ using document root <none> and password file <none>
I20260619 20:35:44.791702 23578 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:44.791746 23578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:44.791890 23578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:44.792936 23578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "71036a0307b343fcaeaa06f4db5bc1a6"
format_stamp: "Formatted at 2026-06-19 20:35:44 on dist-test-slave-4m58"
server_key: "c1c544c2f03b65cf44003bb49be6dab3"
server_key_iv: "10a4efb1981ee50153286b05ae990c00"
server_key_version: "encryptionkey@0"
I20260619 20:35:44.793249 23578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "71036a0307b343fcaeaa06f4db5bc1a6"
format_stamp: "Formatted at 2026-06-19 20:35:44 on dist-test-slave-4m58"
server_key: "c1c544c2f03b65cf44003bb49be6dab3"
server_key_iv: "10a4efb1981ee50153286b05ae990c00"
server_key_version: "encryptionkey@0"
I20260619 20:35:44.794564 23578 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:35:44.795310 23593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:44.795480 23578 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:35:44.795555 23578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "71036a0307b343fcaeaa06f4db5bc1a6"
format_stamp: "Formatted at 2026-06-19 20:35:44 on dist-test-slave-4m58"
server_key: "c1c544c2f03b65cf44003bb49be6dab3"
server_key_iv: "10a4efb1981ee50153286b05ae990c00"
server_key_version: "encryptionkey@0"
I20260619 20:35:44.795621 23578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:44.821484 23578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:44.823451 23578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:44.823570 23578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:44.827260 23578 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:37835
I20260619 20:35:44.827315 23645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:37835 every 8 connection(s)
I20260619 20:35:44.827631 23578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:35:44.828465 23646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:44.830715 23646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71036a0307b343fcaeaa06f4db5bc1a6: Bootstrap starting.
I20260619 20:35:44.831267 23646 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71036a0307b343fcaeaa06f4db5bc1a6: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:44.831526 23646 log.cc:824] T 00000000000000000000000000000000 P 71036a0307b343fcaeaa06f4db5bc1a6: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:44.832284 23646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71036a0307b343fcaeaa06f4db5bc1a6: No bootstrap required, opened a new log
I20260619 20:35:44.833877 23646 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71036a0307b343fcaeaa06f4db5bc1a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71036a0307b343fcaeaa06f4db5bc1a6" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37835 } }
I20260619 20:35:44.834022 23646 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71036a0307b343fcaeaa06f4db5bc1a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:44.834054 23646 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71036a0307b343fcaeaa06f4db5bc1a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71036a0307b343fcaeaa06f4db5bc1a6, State: Initialized, Role: FOLLOWER
I20260619 20:35:44.834184 23646 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71036a0307b343fcaeaa06f4db5bc1a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71036a0307b343fcaeaa06f4db5bc1a6" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37835 } }
I20260619 20:35:44.834244 23646 raft_consensus.cc:388] T 00000000000000000000000000000000 P 71036a0307b343fcaeaa06f4db5bc1a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:35:44.834272 23646 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71036a0307b343fcaeaa06f4db5bc1a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:35:44.834321 23646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71036a0307b343fcaeaa06f4db5bc1a6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:44.834403 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 23578
I20260619 20:35:44.834509 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:35:44.834673 4429 external_mini_cluster.cc:1428] Setting key ebef6ee8da114fe56e2a119eb1ccf099
I20260619 20:35:44.834998 23646 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71036a0307b343fcaeaa06f4db5bc1a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71036a0307b343fcaeaa06f4db5bc1a6" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37835 } }
I20260619 20:35:44.835116 23646 leader_election.cc:302] T 00000000000000000000000000000000 P 71036a0307b343fcaeaa06f4db5bc1a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 71036a0307b343fcaeaa06f4db5bc1a6; no voters:
I20260619 20:35:44.835317 23646 leader_election.cc:288] T 00000000000000000000000000000000 P 71036a0307b343fcaeaa06f4db5bc1a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:35:44.835340 23651 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71036a0307b343fcaeaa06f4db5bc1a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:35:44.835485 23651 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71036a0307b343fcaeaa06f4db5bc1a6 [term 1 LEADER]: Becoming Leader. State: Replica: 71036a0307b343fcaeaa06f4db5bc1a6, State: Running, Role: LEADER
I20260619 20:35:44.835613 23646 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71036a0307b343fcaeaa06f4db5bc1a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:35:44.835587 23651 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71036a0307b343fcaeaa06f4db5bc1a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71036a0307b343fcaeaa06f4db5bc1a6" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37835 } }
I20260619 20:35:44.836315 23652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71036a0307b343fcaeaa06f4db5bc1a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71036a0307b343fcaeaa06f4db5bc1a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71036a0307b343fcaeaa06f4db5bc1a6" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37835 } } }
I20260619 20:35:44.836402 23652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71036a0307b343fcaeaa06f4db5bc1a6 [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:44.836802 23653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71036a0307b343fcaeaa06f4db5bc1a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71036a0307b343fcaeaa06f4db5bc1a6. Latest consensus state: current_term: 1 leader_uuid: "71036a0307b343fcaeaa06f4db5bc1a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71036a0307b343fcaeaa06f4db5bc1a6" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37835 } } }
I20260619 20:35:44.836973 23653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71036a0307b343fcaeaa06f4db5bc1a6 [sys.catalog]: This master's current role is: LEADER
W20260619 20:35:44.837375 23667 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 71036a0307b343fcaeaa06f4db5bc1a6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:35:44.837428 23667 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:35:44.837518 23660 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:35:44.837700 23660 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:35:44.839150 23660 catalog_manager.cc:1347] Generated new cluster ID: 70b93b2077a1407f8e6460e9e7a601e8
I20260619 20:35:44.839198 23660 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:35:44 dist-test-slave-4m58 krb5kdc[23565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:35:44.846508 23650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:44.835137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51780 (local address 127.4.83.126:37835)
0619 20:35:44.835638 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:44.835648 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:44.835684 (+ 36us) server_negotiation.cc:407] Connection header received
0619 20:35:44.835792 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:44.835797 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:44.835871 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:44.836040 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:44.836805 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:44.838044 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:44.838999 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:44.839614 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:44.839720 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:44.842472 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:44.842476 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:44.842477 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:35:44.842489 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:44.844929 (+ 2440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:44.845323 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:44.845326 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:44.845326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:44.845447 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:44.845633 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:44.845634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:44.845634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:44.845902 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:44.846045 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:44.846146 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 20:35:44.846294 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":97,"threads_started":1}
I20260619 20:35:44.850000 23660 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:35:44.850469 23660 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:35:44.855032 23660 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71036a0307b343fcaeaa06f4db5bc1a6: Generated new TSK 0
I20260619 20:35:44.855216 23660 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.83.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
Entry for principal kudu/127.4.83.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab.
I20260619 20:35:44.879263 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:0
--local_ip_for_outbound_sockets=127.4.83.65
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:37835
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:35:44.958498 23672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:44.958693 23672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:44.958715 23672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:44.959874 23672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:44.959916 23672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:44.959941 23672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:35:44.961426 23672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.65.keytab
--principal=kudu/127.4.83.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:37835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:44.961663 23672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:44.961885 23672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:35:44.966729 23672 server_base.cc:1034] running on GCE node
W20260619 20:35:44.966703 23677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:44.966681 23678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:44.966729 23680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:44.967159 23672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:44.967393 23672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:44.968582 23672 hybrid_clock.cc:648] HybridClock initialized: now 1781901344968563 us; error 35 us; skew 500 ppm
Jun 19 20:35:44 dist-test-slave-4m58 krb5kdc[23565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901344, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:35:44.971527 23672 init.cc:377] Logged in from keytab as kudu/127.4.83.65@KRBTEST.COM (short username kudu)
I20260619 20:35:44.972087 23672 webserver.cc:460] Webserver started at http://127.4.83.65:43847/ using document root <none> and password file <none>
I20260619 20:35:44.972301 23672 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:44.972344 23672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:44.972473 23672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:44.973526 23672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-0/data/instance:
uuid: "c6d974c5a88e48f0bca2834f436392d8"
format_stamp: "Formatted at 2026-06-19 20:35:44 on dist-test-slave-4m58"
server_key: "7f0b774c5bd910a7fddc799ad059a24f"
server_key_iv: "5b653d555ce6263075d7927c1cd0fd59"
server_key_version: "encryptionkey@0"
I20260619 20:35:44.973831 23672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance:
uuid: "c6d974c5a88e48f0bca2834f436392d8"
format_stamp: "Formatted at 2026-06-19 20:35:44 on dist-test-slave-4m58"
server_key: "7f0b774c5bd910a7fddc799ad059a24f"
server_key_iv: "5b653d555ce6263075d7927c1cd0fd59"
server_key_version: "encryptionkey@0"
I20260619 20:35:44.975071 23672 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:35:44.975740 23687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:44.975916 23672 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:35:44.975998 23672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "c6d974c5a88e48f0bca2834f436392d8"
format_stamp: "Formatted at 2026-06-19 20:35:44 on dist-test-slave-4m58"
server_key: "7f0b774c5bd910a7fddc799ad059a24f"
server_key_iv: "5b653d555ce6263075d7927c1cd0fd59"
server_key_version: "encryptionkey@0"
I20260619 20:35:44.976061 23672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:45.004839 23672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:45.007323 23672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:45.007475 23672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:45.007710 23672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:35:45.008060 23672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:35:45.008101 23672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:45.008131 23672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:35:45.008147 23672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:45.013612 23672 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:34087
I20260619 20:35:45.013667 23800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:34087 every 8 connection(s)
I20260619 20:35:45.013970 23672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
Jun 19 20:35:45 dist-test-slave-4m58 krb5kdc[23565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901344, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.65@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:35:45.023133 23650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:45.014619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:36075 (local address 127.4.83.126:37835)
0619 20:35:45.014674 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:45.014677 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:45.014847 (+ 170us) server_negotiation.cc:407] Connection header received
0619 20:35:45.015360 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:45.015361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:45.015378 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:45.015429 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:45.016458 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.016967 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.018006 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.018216 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.018274 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:45.020490 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:45.020492 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:45.020492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:35:45.020499 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:45.021769 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:45.022163 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:45.022164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:45.022165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:45.022217 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:45.022443 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:45.022444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:45.022445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:45.022561 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:45.022618 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:45.022926 (+ 308us) server_negotiation.cc:299] Negotiation successful
0619 20:35:45.023030 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 20:35:45.023305 23803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:45.014585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:37835 (local address 127.4.83.65:36075)
0619 20:35:45.014725 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:45.014763 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:45.015258 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:45.015498 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:45.015503 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:45.016140 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:35:45.016344 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:45.016348 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.017060 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.017061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:45.017900 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:45.017902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.018297 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.018298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:45.018426 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:45.018968 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:35:45.018977 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:35:45.020369 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:35:45.021857 (+ 1488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:35:45.021864 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:35:45.021867 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:35:45.022083 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:35:45.022272 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:35:45.022273 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:35:45.022273 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:35:45.022381 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:35:45.022617 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:35:45.022623 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:35:45.022765 (+ 142us) client_negotiation.cc:769] Sending connection context
0619 20:35:45.022891 (+ 126us) client_negotiation.cc:240] Negotiation successful
0619 20:35:45.023061 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":70,"threads_started":1}
I20260619 20:35:45.023856 23801 heartbeater.cc:347] Connected to a master server at 127.4.83.126:37835
I20260619 20:35:45.023958 23801 heartbeater.cc:464] Registering TS with master...
I20260619 20:35:45.024140 23801 heartbeater.cc:511] Master 127.4.83.126:37835 requested a full tablet report, sending...
I20260619 20:35:45.024170 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 23672
I20260619 20:35:45.024281 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance
I20260619 20:35:45.024511 4429 external_mini_cluster.cc:1428] Setting key 55215d6671f33a8dd7f653b0fa738865
I20260619 20:35:45.024696 23610 ts_manager.cc:194] Registered new tserver with Master: c6d974c5a88e48f0bca2834f436392d8 (127.4.83.65:34087)
I20260619 20:35:45.025585 23610 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.65@KRBTEST.COM'} at 127.4.83.65:36075
WARNING: no policy specified for kudu/127.4.83.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
Entry for principal kudu/127.4.83.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab.
I20260619 20:35:45.052242 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:0
--local_ip_for_outbound_sockets=127.4.83.66
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:37835
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:35:45.132717 23806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:45.132941 23806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:45.132972 23806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:45.134133 23806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:45.134178 23806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:45.134213 23806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:35:45.135703 23806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.66.keytab
--principal=kudu/127.4.83.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:37835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:45.135986 23806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:45.136201 23806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:45.140945 23811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:45.140957 23814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:45.140954 23812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:45.141126 23806 server_base.cc:1034] running on GCE node
I20260619 20:35:45.141301 23806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:45.141531 23806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:45.142673 23806 hybrid_clock.cc:648] HybridClock initialized: now 1781901345142660 us; error 26 us; skew 500 ppm
Jun 19 20:35:45 dist-test-slave-4m58 krb5kdc[23565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901345, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:35:45.144950 23806 init.cc:377] Logged in from keytab as kudu/127.4.83.66@KRBTEST.COM (short username kudu)
I20260619 20:35:45.145395 23806 webserver.cc:460] Webserver started at http://127.4.83.66:40985/ using document root <none> and password file <none>
I20260619 20:35:45.145605 23806 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:45.145653 23806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:45.145773 23806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:45.146804 23806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-1/data/instance:
uuid: "db9be827cc8c499f8b1e679e1f2106d3"
format_stamp: "Formatted at 2026-06-19 20:35:45 on dist-test-slave-4m58"
server_key: "b16af38329ef7884fb71dbecff2e3bf2"
server_key_iv: "3b406fe4b93a64ae8d16ae7af4147cc9"
server_key_version: "encryptionkey@0"
I20260619 20:35:45.147125 23806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance:
uuid: "db9be827cc8c499f8b1e679e1f2106d3"
format_stamp: "Formatted at 2026-06-19 20:35:45 on dist-test-slave-4m58"
server_key: "b16af38329ef7884fb71dbecff2e3bf2"
server_key_iv: "3b406fe4b93a64ae8d16ae7af4147cc9"
server_key_version: "encryptionkey@0"
I20260619 20:35:45.148440 23806 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:35:45.149211 23821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:45.149381 23806 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:45.149461 23806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "db9be827cc8c499f8b1e679e1f2106d3"
format_stamp: "Formatted at 2026-06-19 20:35:45 on dist-test-slave-4m58"
server_key: "b16af38329ef7884fb71dbecff2e3bf2"
server_key_iv: "3b406fe4b93a64ae8d16ae7af4147cc9"
server_key_version: "encryptionkey@0"
I20260619 20:35:45.149523 23806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:45.162704 23806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:45.164904 23806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:45.165057 23806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:45.165282 23806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:35:45.165575 23806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:35:45.165606 23806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:45.165637 23806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:35:45.165659 23806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:45.171165 23806 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:37841
I20260619 20:35:45.171208 23934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:37841 every 8 connection(s)
I20260619 20:35:45.171511 23806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
I20260619 20:35:45.177237 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 23806
I20260619 20:35:45.177376 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance
I20260619 20:35:45.177546 4429 external_mini_cluster.cc:1428] Setting key 9b40d9a903c552aed15bf1c6d50411d8
Jun 19 20:35:45 dist-test-slave-4m58 krb5kdc[23565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901345, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.66@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:35:45.181387 23650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:45.172312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:38563 (local address 127.4.83.126:37835)
0619 20:35:45.172367 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:45.172369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:45.172539 (+ 170us) server_negotiation.cc:407] Connection header received
0619 20:35:45.173040 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:45.173042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:45.173053 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:45.173125 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:45.174142 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.174658 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.175718 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.175989 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.176052 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:45.178375 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:45.178376 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:45.178377 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:35:45.178385 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:45.179898 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:45.180375 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:45.180377 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:45.180377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:45.180433 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:45.180732 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:45.180733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:45.180734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:45.180862 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:45.180926 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:45.181198 (+ 272us) server_negotiation.cc:299] Negotiation successful
0619 20:35:45.181295 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 20:35:45.181553 23937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:45.172261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:37835 (local address 127.4.83.66:38563)
0619 20:35:45.172419 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:45.172459 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:45.172932 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:45.173242 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:45.173245 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:45.173815 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:35:45.174022 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:45.174027 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.174781 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.174783 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:45.175621 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:45.175624 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.176070 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.176071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:45.176202 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:45.176696 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:35:45.176704 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:35:45.178278 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:35:45.179972 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:35:45.179980 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:35:45.179983 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:35:45.180297 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:35:45.180505 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:35:45.180505 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:35:45.180506 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:35:45.180646 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:35:45.180921 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:35:45.180926 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:35:45.181069 (+ 143us) client_negotiation.cc:769] Sending connection context
0619 20:35:45.181165 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 20:35:45.181325 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260619 20:35:45.182336 23935 heartbeater.cc:347] Connected to a master server at 127.4.83.126:37835
I20260619 20:35:45.182427 23935 heartbeater.cc:464] Registering TS with master...
I20260619 20:35:45.182606 23935 heartbeater.cc:511] Master 127.4.83.126:37835 requested a full tablet report, sending...
I20260619 20:35:45.182986 23610 ts_manager.cc:194] Registered new tserver with Master: db9be827cc8c499f8b1e679e1f2106d3 (127.4.83.66:37841)
I20260619 20:35:45.183494 23610 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.66@KRBTEST.COM'} at 127.4.83.66:38563
WARNING: no policy specified for kudu/127.4.83.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
Entry for principal kudu/127.4.83.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab.
I20260619 20:35:45.202549 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:0
--local_ip_for_outbound_sockets=127.4.83.67
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:37835
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:35:45.282670 23940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:45.282886 23940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:45.282910 23940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:45.284061 23940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:45.284099 23940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:45.284125 23940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:35:45.285549 23940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.67.keytab
--principal=kudu/127.4.83.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:37835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:45.285755 23940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:45.285955 23940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:45.290762 23945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:45.290761 23946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:45.290756 23948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:45.290998 23940 server_base.cc:1034] running on GCE node
I20260619 20:35:45.291177 23940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:45.291373 23940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:45.292532 23940 hybrid_clock.cc:648] HybridClock initialized: now 1781901345292512 us; error 34 us; skew 500 ppm
Jun 19 20:35:45 dist-test-slave-4m58 krb5kdc[23565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901345, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:35:45.295022 23940 init.cc:377] Logged in from keytab as kudu/127.4.83.67@KRBTEST.COM (short username kudu)
I20260619 20:35:45.295513 23940 webserver.cc:460] Webserver started at http://127.4.83.67:45081/ using document root <none> and password file <none>
I20260619 20:35:45.295712 23940 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:45.295754 23940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:45.295900 23940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:45.296939 23940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-2/data/instance:
uuid: "defd7b6558924b15a3b0d6ec179a7e1a"
format_stamp: "Formatted at 2026-06-19 20:35:45 on dist-test-slave-4m58"
server_key: "866119b12a3b22a948f51c2b8e459bbd"
server_key_iv: "8fdff4965baf8008798d9499859e0417"
server_key_version: "encryptionkey@0"
I20260619 20:35:45.297246 23940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance:
uuid: "defd7b6558924b15a3b0d6ec179a7e1a"
format_stamp: "Formatted at 2026-06-19 20:35:45 on dist-test-slave-4m58"
server_key: "866119b12a3b22a948f51c2b8e459bbd"
server_key_iv: "8fdff4965baf8008798d9499859e0417"
server_key_version: "encryptionkey@0"
I20260619 20:35:45.298506 23940 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:35:45.299206 23955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:45.299382 23940 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:35:45.299474 23940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "defd7b6558924b15a3b0d6ec179a7e1a"
format_stamp: "Formatted at 2026-06-19 20:35:45 on dist-test-slave-4m58"
server_key: "866119b12a3b22a948f51c2b8e459bbd"
server_key_iv: "8fdff4965baf8008798d9499859e0417"
server_key_version: "encryptionkey@0"
I20260619 20:35:45.299542 23940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:45.308851 23940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:45.311161 23940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:45.311287 23940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:45.311542 23940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:35:45.311908 23940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:35:45.311944 23940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:45.311975 23940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:35:45.311991 23940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:45.317580 23940 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:44959
I20260619 20:35:45.317636 24068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:44959 every 8 connection(s)
I20260619 20:35:45.317963 23940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
Jun 19 20:35:45 dist-test-slave-4m58 krb5kdc[23565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901345, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.67@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:35:45.326377 23650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:45.318675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:48575 (local address 127.4.83.126:37835)
0619 20:35:45.318755 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:45.318757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:45.318920 (+ 163us) server_negotiation.cc:407] Connection header received
0619 20:35:45.319400 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:45.319401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:45.319414 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:45.319467 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:45.320438 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.320973 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.321993 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.322179 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.322229 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:45.324270 (+ 2041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:45.324272 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:45.324272 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:35:45.324279 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:45.325348 (+ 1069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:45.325681 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:45.325681 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:45.325682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:45.325726 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:45.325901 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:45.325901 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:45.325902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:45.326001 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:45.326046 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:45.326250 (+ 204us) server_negotiation.cc:299] Negotiation successful
0619 20:35:45.326308 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 20:35:45.326550 24071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:45.318636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:37835 (local address 127.4.83.67:48575)
0619 20:35:45.318782 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:45.318819 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:45.319308 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:45.319529 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:45.319532 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:45.320147 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:35:45.320348 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:45.320352 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.321075 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.321076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:45.321918 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:45.321920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.322237 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.322237 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:45.322336 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:45.322792 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:35:45.322800 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:35:45.324178 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:35:45.325420 (+ 1242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:35:45.325427 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:35:45.325428 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:35:45.325631 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:35:45.325769 (+ 138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:35:45.325770 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:35:45.325770 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:35:45.325860 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:35:45.326043 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:35:45.326046 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:35:45.326158 (+ 112us) client_negotiation.cc:769] Sending connection context
0619 20:35:45.326231 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 20:35:45.326347 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":67,"threads_started":1}
I20260619 20:35:45.327044 24069 heartbeater.cc:347] Connected to a master server at 127.4.83.126:37835
I20260619 20:35:45.327147 24069 heartbeater.cc:464] Registering TS with master...
I20260619 20:35:45.327323 24069 heartbeater.cc:511] Master 127.4.83.126:37835 requested a full tablet report, sending...
I20260619 20:35:45.327353 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 23940
I20260619 20:35:45.327464 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance
I20260619 20:35:45.327652 4429 external_mini_cluster.cc:1428] Setting key ac4b339b0011088362df3601a46fb197
I20260619 20:35:45.327695 23610 ts_manager.cc:194] Registered new tserver with Master: defd7b6558924b15a3b0d6ec179a7e1a (127.4.83.67:44959)
I20260619 20:35:45.328193 23610 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.83.67@KRBTEST.COM'} at 127.4.83.67:48575
I20260619 20:35:45.328472 4429 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:35:45.335448 23650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:45.329704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51792 (local address 127.4.83.126:37835)
0619 20:35:45.329741 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:45.329743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:45.329833 (+ 90us) server_negotiation.cc:407] Connection header received
0619 20:35:45.329880 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:45.329880 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:45.329888 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:45.329928 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:45.330474 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.330954 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.331678 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.331943 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.332006 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:45.333136 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:45.333137 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:45.333137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:35:45.333144 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:45.334424 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:45.334709 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:45.334710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:45.334710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:45.334756 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:45.334915 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:45.334916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:45.334916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:45.335076 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:45.335188 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:45.335313 (+ 125us) server_negotiation.cc:299] Negotiation successful
0619 20:35:45.335380 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 20:35:45.336944 23609 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51792:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:35:45.338969 23609 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:35:45.346658 24078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:45.341579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:37841 (local address 127.0.0.1:47246)
0619 20:35:45.342489 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:45.342497 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:45.342537 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:45.342804 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:45.342804 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:45.342807 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:45.342859 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:45.342861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.344129 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.344130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:45.345308 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:45.345312 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.346354 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.346355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:45.346414 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:45.346441 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 20:35:45.346508 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 20:35:45.346567 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":901,"mutex_wait_us":28,"thread_start_us":36,"threads_started":1}
I20260619 20:35:45.346772 24077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:45.341380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:44959 (local address 127.0.0.1:53570)
0619 20:35:45.342346 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:45.342356 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:45.342417 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:45.342648 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:45.342649 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:45.342651 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:45.342692 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:45.342693 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.343953 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.343954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:45.345290 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:45.345293 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.346525 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.346526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:45.346594 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:45.346599 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:35:45.346672 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 20:35:45.346708 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":950,"mutex_wait_us":7,"thread_start_us":66,"threads_started":1}
I20260619 20:35:45.346843 24081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:45.341683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53570 (local address 127.4.83.67:44959)
0619 20:35:45.341833 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:45.341839 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:45.342415 (+ 576us) server_negotiation.cc:407] Connection header received
0619 20:35:45.342481 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:45.342483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:45.342499 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:45.342566 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:45.342766 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.343815 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.345392 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.346136 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.346186 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:45.346260 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:45.346700 (+ 440us) server_negotiation.cc:299] Negotiation successful
0619 20:35:45.346751 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
I20260619 20:35:45.346879 24079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:45.341380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:34087 (local address 127.0.0.1:54664)
0619 20:35:45.341773 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:45.341851 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:45.341945 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:45.342389 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:45.342400 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:45.342403 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:45.342501 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:45.342504 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.343897 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.343898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:45.345290 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:45.345293 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.346293 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.346294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:45.346594 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:45.346599 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:35:45.346644 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 20:35:45.346695 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":8,"thread_start_us":64,"threads_started":1}
I20260619 20:35:45.346885 24082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:45.341933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47246 (local address 127.4.83.66:37841)
0619 20:35:45.342107 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:45.342111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:45.342590 (+ 479us) server_negotiation.cc:407] Connection header received
0619 20:35:45.342640 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:45.342642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:45.342657 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:45.342721 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:45.342936 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.344015 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.345403 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.346178 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.346245 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:45.346317 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:45.346772 (+ 455us) server_negotiation.cc:299] Negotiation successful
0619 20:35:45.346814 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":75,"threads_started":1}
I20260619 20:35:45.347092 24080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:45.341590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54664 (local address 127.4.83.65:34087)
0619 20:35:45.341975 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:45.341980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:45.341991 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:35:45.342179 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:45.342182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:45.342201 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:45.342284 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:45.342590 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.343758 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.345390 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.346139 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.346199 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:45.346254 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:45.346970 (+ 716us) server_negotiation.cc:299] Negotiation successful
0619 20:35:45.347018 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":97,"threads_started":1}
I20260619 20:35:45.347591 23869 tablet_service.cc:1467] Processing CreateTablet for tablet e6a0faab65da411790070d4415b39b50 (DEFAULT_TABLE table=test-table [id=54d47bbc319d42ff8ffaecfac0fa315f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:35:45.347731 23735 tablet_service.cc:1467] Processing CreateTablet for tablet e6a0faab65da411790070d4415b39b50 (DEFAULT_TABLE table=test-table [id=54d47bbc319d42ff8ffaecfac0fa315f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:35:45.347972 23869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6a0faab65da411790070d4415b39b50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:45.348083 23735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6a0faab65da411790070d4415b39b50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:45.348376 24003 tablet_service.cc:1467] Processing CreateTablet for tablet e6a0faab65da411790070d4415b39b50 (DEFAULT_TABLE table=test-table [id=54d47bbc319d42ff8ffaecfac0fa315f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:35:45.348701 24003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6a0faab65da411790070d4415b39b50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:45.350579 24083 tablet_bootstrap.cc:492] T e6a0faab65da411790070d4415b39b50 P c6d974c5a88e48f0bca2834f436392d8: Bootstrap starting.
I20260619 20:35:45.351121 24083 tablet_bootstrap.cc:654] T e6a0faab65da411790070d4415b39b50 P c6d974c5a88e48f0bca2834f436392d8: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:45.351446 24084 tablet_bootstrap.cc:492] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3: Bootstrap starting.
I20260619 20:35:45.351699 24083 log.cc:824] T e6a0faab65da411790070d4415b39b50 P c6d974c5a88e48f0bca2834f436392d8: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:45.351725 24085 tablet_bootstrap.cc:492] T e6a0faab65da411790070d4415b39b50 P defd7b6558924b15a3b0d6ec179a7e1a: Bootstrap starting.
I20260619 20:35:45.352073 24084 tablet_bootstrap.cc:654] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:45.352375 24085 tablet_bootstrap.cc:654] T e6a0faab65da411790070d4415b39b50 P defd7b6558924b15a3b0d6ec179a7e1a: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:45.352615 24084 log.cc:824] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:45.352825 24085 log.cc:824] T e6a0faab65da411790070d4415b39b50 P defd7b6558924b15a3b0d6ec179a7e1a: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:45.353353 24084 tablet_bootstrap.cc:492] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3: No bootstrap required, opened a new log
I20260619 20:35:45.353353 24083 tablet_bootstrap.cc:492] T e6a0faab65da411790070d4415b39b50 P c6d974c5a88e48f0bca2834f436392d8: No bootstrap required, opened a new log
I20260619 20:35:45.353420 24084 ts_tablet_manager.cc:1397] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:35:45.353449 24083 ts_tablet_manager.cc:1397] T e6a0faab65da411790070d4415b39b50 P c6d974c5a88e48f0bca2834f436392d8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 20:35:45.353487 24085 tablet_bootstrap.cc:492] T e6a0faab65da411790070d4415b39b50 P defd7b6558924b15a3b0d6ec179a7e1a: No bootstrap required, opened a new log
I20260619 20:35:45.353546 24085 ts_tablet_manager.cc:1397] T e6a0faab65da411790070d4415b39b50 P defd7b6558924b15a3b0d6ec179a7e1a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:35:45.354965 24085 raft_consensus.cc:348] T e6a0faab65da411790070d4415b39b50 P defd7b6558924b15a3b0d6ec179a7e1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d974c5a88e48f0bca2834f436392d8" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34087 } } peers { permanent_uuid: "defd7b6558924b15a3b0d6ec179a7e1a" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44959 } } peers { permanent_uuid: "db9be827cc8c499f8b1e679e1f2106d3" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37841 } }
I20260619 20:35:45.355106 24085 raft_consensus.cc:374] T e6a0faab65da411790070d4415b39b50 P defd7b6558924b15a3b0d6ec179a7e1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:45.355137 24085 raft_consensus.cc:729] T e6a0faab65da411790070d4415b39b50 P defd7b6558924b15a3b0d6ec179a7e1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: defd7b6558924b15a3b0d6ec179a7e1a, State: Initialized, Role: FOLLOWER
I20260619 20:35:45.355247 24085 consensus_queue.cc:260] T e6a0faab65da411790070d4415b39b50 P defd7b6558924b15a3b0d6ec179a7e1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d974c5a88e48f0bca2834f436392d8" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34087 } } peers { permanent_uuid: "defd7b6558924b15a3b0d6ec179a7e1a" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44959 } } peers { permanent_uuid: "db9be827cc8c499f8b1e679e1f2106d3" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37841 } }
I20260619 20:35:45.355288 24083 raft_consensus.cc:348] T e6a0faab65da411790070d4415b39b50 P c6d974c5a88e48f0bca2834f436392d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d974c5a88e48f0bca2834f436392d8" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34087 } } peers { permanent_uuid: "defd7b6558924b15a3b0d6ec179a7e1a" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44959 } } peers { permanent_uuid: "db9be827cc8c499f8b1e679e1f2106d3" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37841 } }
I20260619 20:35:45.355281 24084 raft_consensus.cc:348] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d974c5a88e48f0bca2834f436392d8" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34087 } } peers { permanent_uuid: "defd7b6558924b15a3b0d6ec179a7e1a" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44959 } } peers { permanent_uuid: "db9be827cc8c499f8b1e679e1f2106d3" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37841 } }
I20260619 20:35:45.355448 24083 raft_consensus.cc:374] T e6a0faab65da411790070d4415b39b50 P c6d974c5a88e48f0bca2834f436392d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:45.355468 24084 raft_consensus.cc:374] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:45.355481 24083 raft_consensus.cc:729] T e6a0faab65da411790070d4415b39b50 P c6d974c5a88e48f0bca2834f436392d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6d974c5a88e48f0bca2834f436392d8, State: Initialized, Role: FOLLOWER
I20260619 20:35:45.355497 24084 raft_consensus.cc:729] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db9be827cc8c499f8b1e679e1f2106d3, State: Initialized, Role: FOLLOWER
I20260619 20:35:45.355507 24085 ts_tablet_manager.cc:1428] T e6a0faab65da411790070d4415b39b50 P defd7b6558924b15a3b0d6ec179a7e1a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:35:45.355602 24083 consensus_queue.cc:260] T e6a0faab65da411790070d4415b39b50 P c6d974c5a88e48f0bca2834f436392d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d974c5a88e48f0bca2834f436392d8" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34087 } } peers { permanent_uuid: "defd7b6558924b15a3b0d6ec179a7e1a" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44959 } } peers { permanent_uuid: "db9be827cc8c499f8b1e679e1f2106d3" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37841 } }
I20260619 20:35:45.355602 24084 consensus_queue.cc:260] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d974c5a88e48f0bca2834f436392d8" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34087 } } peers { permanent_uuid: "defd7b6558924b15a3b0d6ec179a7e1a" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44959 } } peers { permanent_uuid: "db9be827cc8c499f8b1e679e1f2106d3" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37841 } }
I20260619 20:35:45.355649 24069 heartbeater.cc:503] Master 127.4.83.126:37835 was elected leader, sending a full tablet report...
I20260619 20:35:45.355880 23801 heartbeater.cc:503] Master 127.4.83.126:37835 was elected leader, sending a full tablet report...
I20260619 20:35:45.355894 24083 ts_tablet_manager.cc:1428] T e6a0faab65da411790070d4415b39b50 P c6d974c5a88e48f0bca2834f436392d8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:35:45.356101 24084 ts_tablet_manager.cc:1428] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:35:45.356204 23935 heartbeater.cc:503] Master 127.4.83.126:37835 was elected leader, sending a full tablet report...
I20260619 20:35:45.408965 24091 raft_consensus.cc:482] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:35:45.409119 24091 raft_consensus.cc:504] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d974c5a88e48f0bca2834f436392d8" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34087 } } peers { permanent_uuid: "defd7b6558924b15a3b0d6ec179a7e1a" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44959 } } peers { permanent_uuid: "db9be827cc8c499f8b1e679e1f2106d3" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37841 } }
I20260619 20:35:45.409479 24091 leader_election.cc:288] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6d974c5a88e48f0bca2834f436392d8 (127.4.83.65:34087), defd7b6558924b15a3b0d6ec179a7e1a (127.4.83.67:44959)
I20260619 20:35:45.412982 24081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:45.409772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:52613 (local address 127.4.83.67:44959)
0619 20:35:45.409868 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:45.409870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:45.410012 (+ 142us) server_negotiation.cc:407] Connection header received
0619 20:35:45.410047 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:45.410048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:45.410065 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:45.410134 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:45.410362 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.410853 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.412239 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.412641 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.412703 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:45.412716 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:45.412868 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 20:35:45.412911 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 20:35:45.412992 24080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:45.409774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:34657 (local address 127.4.83.65:34087)
0619 20:35:45.409825 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:45.409827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:45.409837 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:35:45.409908 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:45.409909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:45.409921 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:45.409983 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:45.410211 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.410755 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.412274 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.412655 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.412720 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:45.412732 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:45.412884 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 20:35:45.412924 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 20:35:45.413008 23937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:45.409707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:34087 (local address 127.4.83.66:34657)
0619 20:35:45.409750 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:45.409762 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:45.409827 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:45.410028 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:45.410029 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:45.410031 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:45.410120 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:45.410122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.410853 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.410854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:45.412160 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:45.412162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.412738 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.412739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:45.412813 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:45.412816 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:35:45.412867 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 20:35:45.412912 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21,"mutex_wait_us":5}
I20260619 20:35:45.413013 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:45.409707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:44959 (local address 127.4.83.66:52613)
0619 20:35:45.409936 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:45.409944 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:45.409991 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:45.410180 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:45.410182 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:45.410184 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:45.410263 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:45.410264 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.410945 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.410945 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:45.412159 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:45.412162 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.412724 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.412725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:45.412800 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:45.412804 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:35:45.412845 (+ 41us) client_negotiation.cc:240] Negotiation successful
0619 20:35:45.412891 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":92,"thread_start_us":113,"threads_started":1}
I20260619 20:35:45.413407 24023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6a0faab65da411790070d4415b39b50" candidate_uuid: "db9be827cc8c499f8b1e679e1f2106d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "defd7b6558924b15a3b0d6ec179a7e1a" is_pre_election: true
I20260619 20:35:45.413414 23755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6a0faab65da411790070d4415b39b50" candidate_uuid: "db9be827cc8c499f8b1e679e1f2106d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6d974c5a88e48f0bca2834f436392d8" is_pre_election: true
I20260619 20:35:45.413551 24023 raft_consensus.cc:2454] T e6a0faab65da411790070d4415b39b50 P defd7b6558924b15a3b0d6ec179a7e1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db9be827cc8c499f8b1e679e1f2106d3 in term 0.
I20260619 20:35:45.413566 23755 raft_consensus.cc:2454] T e6a0faab65da411790070d4415b39b50 P c6d974c5a88e48f0bca2834f436392d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db9be827cc8c499f8b1e679e1f2106d3 in term 0.
I20260619 20:35:45.413793 23825 leader_election.cc:302] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db9be827cc8c499f8b1e679e1f2106d3, defd7b6558924b15a3b0d6ec179a7e1a; no voters:
I20260619 20:35:45.413933 24091 raft_consensus.cc:2781] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:35:45.413992 24091 raft_consensus.cc:482] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:35:45.414007 24091 raft_consensus.cc:3037] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:45.414675 24091 raft_consensus.cc:504] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d974c5a88e48f0bca2834f436392d8" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34087 } } peers { permanent_uuid: "defd7b6558924b15a3b0d6ec179a7e1a" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44959 } } peers { permanent_uuid: "db9be827cc8c499f8b1e679e1f2106d3" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37841 } }
I20260619 20:35:45.414794 24091 leader_election.cc:288] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3 [CANDIDATE]: Term 1 election: Requested vote from peers c6d974c5a88e48f0bca2834f436392d8 (127.4.83.65:34087), defd7b6558924b15a3b0d6ec179a7e1a (127.4.83.67:44959)
I20260619 20:35:45.414948 24023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6a0faab65da411790070d4415b39b50" candidate_uuid: "db9be827cc8c499f8b1e679e1f2106d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "defd7b6558924b15a3b0d6ec179a7e1a"
I20260619 20:35:45.414955 23755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6a0faab65da411790070d4415b39b50" candidate_uuid: "db9be827cc8c499f8b1e679e1f2106d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6d974c5a88e48f0bca2834f436392d8"
I20260619 20:35:45.415023 23755 raft_consensus.cc:3037] T e6a0faab65da411790070d4415b39b50 P c6d974c5a88e48f0bca2834f436392d8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:45.415022 24023 raft_consensus.cc:3037] T e6a0faab65da411790070d4415b39b50 P defd7b6558924b15a3b0d6ec179a7e1a [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:45.415699 23755 raft_consensus.cc:2454] T e6a0faab65da411790070d4415b39b50 P c6d974c5a88e48f0bca2834f436392d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db9be827cc8c499f8b1e679e1f2106d3 in term 1.
I20260619 20:35:45.415701 24023 raft_consensus.cc:2454] T e6a0faab65da411790070d4415b39b50 P defd7b6558924b15a3b0d6ec179a7e1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db9be827cc8c499f8b1e679e1f2106d3 in term 1.
I20260619 20:35:45.415866 23825 leader_election.cc:302] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db9be827cc8c499f8b1e679e1f2106d3, defd7b6558924b15a3b0d6ec179a7e1a; no voters:
I20260619 20:35:45.415970 24091 raft_consensus.cc:2781] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:35:45.416126 24091 raft_consensus.cc:686] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3 [term 1 LEADER]: Becoming Leader. State: Replica: db9be827cc8c499f8b1e679e1f2106d3, State: Running, Role: LEADER
I20260619 20:35:45.416225 24091 consensus_queue.cc:237] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d974c5a88e48f0bca2834f436392d8" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34087 } } peers { permanent_uuid: "defd7b6558924b15a3b0d6ec179a7e1a" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44959 } } peers { permanent_uuid: "db9be827cc8c499f8b1e679e1f2106d3" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37841 } }
I20260619 20:35:45.417126 23609 catalog_manager.cc:5495] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to db9be827cc8c499f8b1e679e1f2106d3 (127.4.83.66). New cstate: current_term: 1 leader_uuid: "db9be827cc8c499f8b1e679e1f2106d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6d974c5a88e48f0bca2834f436392d8" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "defd7b6558924b15a3b0d6ec179a7e1a" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db9be827cc8c499f8b1e679e1f2106d3" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 37841 } health_report { overall_health: HEALTHY } } }
W20260619 20:35:45.422068 23936 tablet.cc:2369] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:35:45.439558 24082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:45.437166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47250 (local address 127.4.83.66:37841)
0619 20:35:45.437231 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:45.437233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:45.437240 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:35:45.437277 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:45.437278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:45.437299 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:45.437355 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:35:45.437466 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.437953 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.438766 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.438983 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.439041 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:45.439272 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:35:45.439384 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:35:45.439459 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:45.439472 (+ 13us) server_negotiation.cc:299] Negotiation successful
0619 20:35:45.439494 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 20:35:45.441376 23755 raft_consensus.cc:1261] T e6a0faab65da411790070d4415b39b50 P c6d974c5a88e48f0bca2834f436392d8 [term 1 FOLLOWER]: Refusing update from remote peer db9be827cc8c499f8b1e679e1f2106d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:35:45.441373 24023 raft_consensus.cc:1261] T e6a0faab65da411790070d4415b39b50 P defd7b6558924b15a3b0d6ec179a7e1a [term 1 FOLLOWER]: Refusing update from remote peer db9be827cc8c499f8b1e679e1f2106d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:35:45.441735 24093 consensus_queue.cc:1035] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "defd7b6558924b15a3b0d6ec179a7e1a" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 44959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:35:45.441856 24091 consensus_queue.cc:1035] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6d974c5a88e48f0bca2834f436392d8" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 34087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:35:45.446493 23609 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:35:45.446589 23609 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:35:45.447270 23609 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 71036a0307b343fcaeaa06f4db5bc1a6: Sending DeleteTablet for 3 replicas of tablet e6a0faab65da411790070d4415b39b50
I20260619 20:35:45.447602 23735 tablet_service.cc:1514] Processing DeleteTablet for tablet e6a0faab65da411790070d4415b39b50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:35:45 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:54664
I20260619 20:35:45.447615 23869 tablet_service.cc:1514] Processing DeleteTablet for tablet e6a0faab65da411790070d4415b39b50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:35:45 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:47246
I20260619 20:35:45.447753 24003 tablet_service.cc:1514] Processing DeleteTablet for tablet e6a0faab65da411790070d4415b39b50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:35:45 UTC) from {username='kudu', principal='kudu/127.4.83.126@KRBTEST.COM'} at 127.0.0.1:53570
I20260619 20:35:45.447896 24106 tablet_replica.cc:330] T e6a0faab65da411790070d4415b39b50 P c6d974c5a88e48f0bca2834f436392d8: stopping tablet replica
I20260619 20:35:45.448010 24106 raft_consensus.cc:2229] T e6a0faab65da411790070d4415b39b50 P c6d974c5a88e48f0bca2834f436392d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:35:45.448192 24106 raft_consensus.cc:2258] T e6a0faab65da411790070d4415b39b50 P c6d974c5a88e48f0bca2834f436392d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:35:45.448386 24107 tablet_replica.cc:330] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3: stopping tablet replica
I20260619 20:35:45.448415 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 23672
I20260619 20:35:45.449440 24107 raft_consensus.cc:2229] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:35:45.449584 24107 raft_consensus.cc:2258] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:35:45.450032 24107 ts_tablet_manager.cc:1905] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:35:45.451814 24107 ts_tablet_manager.cc:1918] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:35:45.451920 24107 log.cc:1196] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-1/wal/wals/e6a0faab65da411790070d4415b39b50
I20260619 20:35:45.452119 24107 ts_tablet_manager.cc:1939] T e6a0faab65da411790070d4415b39b50 P db9be827cc8c499f8b1e679e1f2106d3: Deleting consensus metadata
I20260619 20:35:45.452533 23596 catalog_manager.cc:4841] TS db9be827cc8c499f8b1e679e1f2106d3 (127.4.83.66:37841): tablet e6a0faab65da411790070d4415b39b50 (table test-table [id=54d47bbc319d42ff8ffaecfac0fa315f]) successfully deleted
W20260619 20:35:45.455717 23596 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.83.65:34087 (error 108)
I20260619 20:35:45.456036 24108 tablet_replica.cc:330] T e6a0faab65da411790070d4415b39b50 P defd7b6558924b15a3b0d6ec179a7e1a: stopping tablet replica
I20260619 20:35:45.456156 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 23806
I20260619 20:35:45.456180 24108 raft_consensus.cc:2229] T e6a0faab65da411790070d4415b39b50 P defd7b6558924b15a3b0d6ec179a7e1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:35:45.456311 24108 raft_consensus.cc:2258] T e6a0faab65da411790070d4415b39b50 P defd7b6558924b15a3b0d6ec179a7e1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:35:45.456342 24079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:45.456217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:34087 (local address 127.0.0.1:54674)
0619 20:35:45.456272 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:45.456306 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.65:34087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 20:35:45.456810 23596 catalog_manager.cc:4584] TS c6d974c5a88e48f0bca2834f436392d8 (127.4.83.65:34087): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e6a0faab65da411790070d4415b39b50: Network error: Client connection negotiation failed: client connection to 127.4.83.65:34087: connect: Connection refused (error 111)
I20260619 20:35:45.458003 24108 ts_tablet_manager.cc:1905] T e6a0faab65da411790070d4415b39b50 P defd7b6558924b15a3b0d6ec179a7e1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:35:45.459362 24108 ts_tablet_manager.cc:1918] T e6a0faab65da411790070d4415b39b50 P defd7b6558924b15a3b0d6ec179a7e1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:35:45.459417 24108 log.cc:1196] T e6a0faab65da411790070d4415b39b50 P defd7b6558924b15a3b0d6ec179a7e1a: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901102012824-4429-0/minicluster-data/ts-2/wal/wals/e6a0faab65da411790070d4415b39b50
I20260619 20:35:45.459604 24108 ts_tablet_manager.cc:1939] T e6a0faab65da411790070d4415b39b50 P defd7b6558924b15a3b0d6ec179a7e1a: Deleting consensus metadata
I20260619 20:35:45.460085 23597 catalog_manager.cc:4841] TS defd7b6558924b15a3b0d6ec179a7e1a (127.4.83.67:44959): tablet e6a0faab65da411790070d4415b39b50 (table test-table [id=54d47bbc319d42ff8ffaecfac0fa315f]) successfully deleted
I20260619 20:35:45.460783 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 23940
I20260619 20:35:45.465075 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 23578
2026-06-19T20:35:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1853 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-19T20:35:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:35:45Z Disabled control of system clock
I20260619 20:35:45.492893 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:45309
--webserver_interface=127.4.83.126
--webserver_port=0
--builtin_ntp_servers=127.4.83.84:44095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:45309
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 20:35:45.575177 24113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:45.575448 24113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:45.575471 24113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:45.576968 24113 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:35:45.577034 24113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:45.577047 24113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:45.577060 24113 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:35:45.577071 24113 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:35:45.578601 24113 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:44095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:45309
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:45309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:45.578856 24113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:45.579126 24113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:45.583870 24121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:45.583871 24119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:45.583870 24118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:45.584086 24113 server_base.cc:1034] running on GCE node
I20260619 20:35:45.584432 24113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:45.584729 24113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:45.585891 24113 hybrid_clock.cc:648] HybridClock initialized: now 1781901345585867 us; error 47 us; skew 500 ppm
I20260619 20:35:45.587381 24113 webserver.cc:460] Webserver started at http://127.4.83.126:36417/ using document root <none> and password file <none>
I20260619 20:35:45.587635 24113 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:45.587684 24113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:45.587800 24113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:45.588943 24113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "7201b746b8fa4816b95c6dc1d9c4c0f4"
format_stamp: "Formatted at 2026-06-19 20:35:45 on dist-test-slave-4m58"
server_key: "ef2cf68d129bc5c439ffe77cd4b42f90"
server_key_iv: "9c47513258b0a076ed095d48307fb810"
server_key_version: "encryptionkey@0"
I20260619 20:35:45.589278 24113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "7201b746b8fa4816b95c6dc1d9c4c0f4"
format_stamp: "Formatted at 2026-06-19 20:35:45 on dist-test-slave-4m58"
server_key: "ef2cf68d129bc5c439ffe77cd4b42f90"
server_key_iv: "9c47513258b0a076ed095d48307fb810"
server_key_version: "encryptionkey@0"
I20260619 20:35:45.590566 24113 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:35:45.591382 24127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:45.591588 24113 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:45.591681 24113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "7201b746b8fa4816b95c6dc1d9c4c0f4"
format_stamp: "Formatted at 2026-06-19 20:35:45 on dist-test-slave-4m58"
server_key: "ef2cf68d129bc5c439ffe77cd4b42f90"
server_key_iv: "9c47513258b0a076ed095d48307fb810"
server_key_version: "encryptionkey@0"
I20260619 20:35:45.591745 24113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:45.598717 24113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:45.599047 24113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:45.599174 24113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:45.602837 24113 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:45309
I20260619 20:35:45.602880 24179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:45309 every 8 connection(s)
I20260619 20:35:45.603204 24113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:35:45.604188 24180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:45.606894 24180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7201b746b8fa4816b95c6dc1d9c4c0f4: Bootstrap starting.
I20260619 20:35:45.607466 24180 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7201b746b8fa4816b95c6dc1d9c4c0f4: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:45.607535 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 24113
I20260619 20:35:45.607649 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:35:45.607864 4429 external_mini_cluster.cc:1428] Setting key c506dca738b1efee13d5cd56fe9e05ba
I20260619 20:35:45.607896 24180 log.cc:824] T 00000000000000000000000000000000 P 7201b746b8fa4816b95c6dc1d9c4c0f4: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:45.608673 24180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7201b746b8fa4816b95c6dc1d9c4c0f4: No bootstrap required, opened a new log
I20260619 20:35:45.611330 24180 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7201b746b8fa4816b95c6dc1d9c4c0f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7201b746b8fa4816b95c6dc1d9c4c0f4" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45309 } }
I20260619 20:35:45.611488 24180 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7201b746b8fa4816b95c6dc1d9c4c0f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:45.611521 24180 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7201b746b8fa4816b95c6dc1d9c4c0f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7201b746b8fa4816b95c6dc1d9c4c0f4, State: Initialized, Role: FOLLOWER
I20260619 20:35:45.611647 24180 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7201b746b8fa4816b95c6dc1d9c4c0f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7201b746b8fa4816b95c6dc1d9c4c0f4" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45309 } }
I20260619 20:35:45.611723 24180 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7201b746b8fa4816b95c6dc1d9c4c0f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:35:45.611754 24180 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7201b746b8fa4816b95c6dc1d9c4c0f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:35:45.611820 24180 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7201b746b8fa4816b95c6dc1d9c4c0f4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:45.612637 24180 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7201b746b8fa4816b95c6dc1d9c4c0f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7201b746b8fa4816b95c6dc1d9c4c0f4" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45309 } }
I20260619 20:35:45.612752 24180 leader_election.cc:302] T 00000000000000000000000000000000 P 7201b746b8fa4816b95c6dc1d9c4c0f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7201b746b8fa4816b95c6dc1d9c4c0f4; no voters:
I20260619 20:35:45.612949 24180 leader_election.cc:288] T 00000000000000000000000000000000 P 7201b746b8fa4816b95c6dc1d9c4c0f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:35:45.613041 24185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7201b746b8fa4816b95c6dc1d9c4c0f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:35:45.613183 24185 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7201b746b8fa4816b95c6dc1d9c4c0f4 [term 1 LEADER]: Becoming Leader. State: Replica: 7201b746b8fa4816b95c6dc1d9c4c0f4, State: Running, Role: LEADER
I20260619 20:35:45.613232 24180 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7201b746b8fa4816b95c6dc1d9c4c0f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:35:45.613306 24185 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7201b746b8fa4816b95c6dc1d9c4c0f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7201b746b8fa4816b95c6dc1d9c4c0f4" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45309 } }
I20260619 20:35:45.613684 24185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7201b746b8fa4816b95c6dc1d9c4c0f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7201b746b8fa4816b95c6dc1d9c4c0f4. Latest consensus state: current_term: 1 leader_uuid: "7201b746b8fa4816b95c6dc1d9c4c0f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7201b746b8fa4816b95c6dc1d9c4c0f4" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45309 } } }
I20260619 20:35:45.613768 24185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7201b746b8fa4816b95c6dc1d9c4c0f4 [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:45.613968 24187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7201b746b8fa4816b95c6dc1d9c4c0f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7201b746b8fa4816b95c6dc1d9c4c0f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7201b746b8fa4816b95c6dc1d9c4c0f4" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45309 } } }
I20260619 20:35:45.614317 24184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:45.608414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59304 (local address 127.4.83.126:45309)
0619 20:35:45.608614 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:45.608624 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:45.608655 (+ 31us) server_negotiation.cc:407] Connection header received
0619 20:35:45.609184 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:45.609193 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:45.609278 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:45.609459 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:45.609649 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.611067 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.612039 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.612652 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.612781 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:45.613497 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:45.613500 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:45.613501 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:35:45.613699 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:45.613745 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:35:45.613761 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:45.613800 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:45.613935 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 20:35:45.614057 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":72,"threads_started":1}
I20260619 20:35:45.614455 24193 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:35:45.614117 24187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7201b746b8fa4816b95c6dc1d9c4c0f4 [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:45.614954 24193 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:35:45.616559 24193 catalog_manager.cc:1347] Generated new cluster ID: bb226f48b8c34459a37d48b3a15833a2
I20260619 20:35:45.616600 24193 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:35:45.622298 24193 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:35:45.622798 24193 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:35:45.627562 24193 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7201b746b8fa4816b95c6dc1d9c4c0f4: Generated new TSK 0
I20260619 20:35:45.627740 24193 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:35:45.631980 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.65:0
--local_ip_for_outbound_sockets=127.4.83.65
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:45309
--builtin_ntp_servers=127.4.83.84:44095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 20:35:45.712710 24204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:45.712945 24204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:45.712972 24204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:45.714306 24204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:45.714356 24204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:45.714385 24204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.65
I20260619 20:35:45.715919 24204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:44095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.83.65
--webserver_port=0
--tserver_master_addrs=127.4.83.126:45309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.65
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:45.716161 24204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:45.716465 24204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:45.721319 24209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:45.721510 24212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:45.721284 24210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:45.721589 24204 server_base.cc:1034] running on GCE node
I20260619 20:35:45.721828 24204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:45.722108 24204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:45.723279 24204 hybrid_clock.cc:648] HybridClock initialized: now 1781901345723259 us; error 41 us; skew 500 ppm
I20260619 20:35:45.724750 24204 webserver.cc:460] Webserver started at http://127.4.83.65:43863/ using document root <none> and password file <none>
I20260619 20:35:45.725095 24204 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:45.725157 24204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:45.725283 24204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:45.726660 24204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-0/data/instance:
uuid: "746524891023432999cee081e25ed660"
format_stamp: "Formatted at 2026-06-19 20:35:45 on dist-test-slave-4m58"
server_key: "1ac2bf827de7af49ee68250ad92f986f"
server_key_iv: "31bc4ac2d453c0eca0491d7fba9c554d"
server_key_version: "encryptionkey@0"
I20260619 20:35:45.727046 24204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance:
uuid: "746524891023432999cee081e25ed660"
format_stamp: "Formatted at 2026-06-19 20:35:45 on dist-test-slave-4m58"
server_key: "1ac2bf827de7af49ee68250ad92f986f"
server_key_iv: "31bc4ac2d453c0eca0491d7fba9c554d"
server_key_version: "encryptionkey@0"
I20260619 20:35:45.728621 24204 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260619 20:35:45.729496 24218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:45.729665 24204 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:35:45.729754 24204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-0/wal
uuid: "746524891023432999cee081e25ed660"
format_stamp: "Formatted at 2026-06-19 20:35:45 on dist-test-slave-4m58"
server_key: "1ac2bf827de7af49ee68250ad92f986f"
server_key_iv: "31bc4ac2d453c0eca0491d7fba9c554d"
server_key_version: "encryptionkey@0"
I20260619 20:35:45.729835 24204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:45.742587 24204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:45.743017 24204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:45.743160 24204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:45.743391 24204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:35:45.743729 24204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:35:45.743779 24204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:45.743814 24204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:35:45.743861 24204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:45.749936 24204 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.65:41119
I20260619 20:35:45.749991 24331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.65:41119 every 8 connection(s)
I20260619 20:35:45.750296 24204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-0/data/info.pb
I20260619 20:35:45.755257 24184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:45.750934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:48689 (local address 127.4.83.126:45309)
0619 20:35:45.750978 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:45.750980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:45.751207 (+ 227us) server_negotiation.cc:407] Connection header received
0619 20:35:45.751708 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:45.751709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:45.751722 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:45.751781 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:45.752465 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.752964 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.753909 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.754112 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.754167 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:45.755001 (+ 834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:45.755002 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:45.755002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:35:45.755007 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:45.755022 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:35:45.755027 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:45.755071 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:45.755150 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 20:35:45.755177 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 20:35:45.755465 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:45.750928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:45309 (local address 127.4.83.65:48689)
0619 20:35:45.751082 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:45.751116 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:45.751606 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:45.751878 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:45.751881 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:45.751889 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:35:45.752373 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:45.752377 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.753049 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.753050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:45.753822 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:45.753825 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.754182 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.754184 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:45.754308 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:45.754855 (+ 547us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:35:45.754855 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:35:45.754885 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:35:45.754911 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:35:45.755079 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:35:45.755084 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:35:45.755087 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:35:45.755145 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 20:35:45.755250 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":70,"threads_started":1}
I20260619 20:35:45.755998 24332 heartbeater.cc:347] Connected to a master server at 127.4.83.126:45309
I20260619 20:35:45.756088 24332 heartbeater.cc:464] Registering TS with master...
I20260619 20:35:45.756233 24332 heartbeater.cc:511] Master 127.4.83.126:45309 requested a full tablet report, sending...
I20260619 20:35:45.756857 24144 ts_manager.cc:194] Registered new tserver with Master: 746524891023432999cee081e25ed660 (127.4.83.65:41119)
I20260619 20:35:45.757161 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 24204
I20260619 20:35:45.757277 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-0/wal/instance
I20260619 20:35:45.757508 4429 external_mini_cluster.cc:1428] Setting key 30e895a857cd8563c4420f20f305b245
I20260619 20:35:45.757715 24144 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.83.65:48689
I20260619 20:35:45.758630 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.66:0
--local_ip_for_outbound_sockets=127.4.83.66
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:45309
--builtin_ntp_servers=127.4.83.84:44095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 20:35:45.839543 24335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:45.839774 24335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:45.839804 24335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:45.841007 24335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:45.841054 24335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:45.841080 24335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.66
I20260619 20:35:45.842440 24335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:44095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.83.66
--webserver_port=0
--tserver_master_addrs=127.4.83.126:45309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.66
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:45.842682 24335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:45.842990 24335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:45.847915 24340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:45.847913 24341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:45.847899 24343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:45.847936 24335 server_base.cc:1034] running on GCE node
I20260619 20:35:45.848336 24335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:45.848601 24335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:45.849771 24335 hybrid_clock.cc:648] HybridClock initialized: now 1781901345849755 us; error 25 us; skew 500 ppm
I20260619 20:35:45.851114 24335 webserver.cc:460] Webserver started at http://127.4.83.66:33847/ using document root <none> and password file <none>
I20260619 20:35:45.851329 24335 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:45.851390 24335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:45.851516 24335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:45.852630 24335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-1/data/instance:
uuid: "eac21ab86d3a4abbbb5c1ed5e88d2cc8"
format_stamp: "Formatted at 2026-06-19 20:35:45 on dist-test-slave-4m58"
server_key: "059137c002bdf8cf39d0d1172e006558"
server_key_iv: "426ba00a915bf75626232d781cf65faa"
server_key_version: "encryptionkey@0"
I20260619 20:35:45.852989 24335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance:
uuid: "eac21ab86d3a4abbbb5c1ed5e88d2cc8"
format_stamp: "Formatted at 2026-06-19 20:35:45 on dist-test-slave-4m58"
server_key: "059137c002bdf8cf39d0d1172e006558"
server_key_iv: "426ba00a915bf75626232d781cf65faa"
server_key_version: "encryptionkey@0"
I20260619 20:35:45.854244 24335 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 20:35:45.854935 24349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:45.855083 24335 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:35:45.855160 24335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-1/wal
uuid: "eac21ab86d3a4abbbb5c1ed5e88d2cc8"
format_stamp: "Formatted at 2026-06-19 20:35:45 on dist-test-slave-4m58"
server_key: "059137c002bdf8cf39d0d1172e006558"
server_key_iv: "426ba00a915bf75626232d781cf65faa"
server_key_version: "encryptionkey@0"
I20260619 20:35:45.855227 24335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:45.879261 24335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:45.879585 24335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:45.879717 24335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:45.879971 24335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:35:45.880297 24335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:35:45.880329 24335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:45.880352 24335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:35:45.880375 24335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:45.885797 24335 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.66:33361
I20260619 20:35:45.885845 24462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.66:33361 every 8 connection(s)
I20260619 20:35:45.886160 24335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-1/data/info.pb
I20260619 20:35:45.891211 24184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:45.886865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.66:36775 (local address 127.4.83.126:45309)
0619 20:35:45.886948 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:45.886950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:45.887158 (+ 208us) server_negotiation.cc:407] Connection header received
0619 20:35:45.887684 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:45.887685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:45.887696 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:45.887753 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:45.888483 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.888988 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.889930 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.890126 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.890188 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:45.890963 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:45.890964 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:45.890964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:35:45.890969 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:45.890984 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:35:45.890987 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:45.891031 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:45.891121 (+ 90us) server_negotiation.cc:299] Negotiation successful
0619 20:35:45.891146 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 20:35:45.891345 24465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:45.886853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:45309 (local address 127.4.83.66:36775)
0619 20:35:45.887044 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:45.887078 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:45.887563 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:45.887817 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:45.887820 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:45.887857 (+ 37us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:35:45.888371 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:45.888376 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.889063 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.889064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:45.889846 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:45.889849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:45.890195 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:45.890196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:45.890299 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:45.890838 (+ 539us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:35:45.890839 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:35:45.890867 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:35:45.890890 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:35:45.891038 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:35:45.891040 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:35:45.891043 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:35:45.891089 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 20:35:45.891170 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":88,"threads_started":1}
I20260619 20:35:45.891788 24463 heartbeater.cc:347] Connected to a master server at 127.4.83.126:45309
I20260619 20:35:45.891903 24463 heartbeater.cc:464] Registering TS with master...
I20260619 20:35:45.892078 24463 heartbeater.cc:511] Master 127.4.83.126:45309 requested a full tablet report, sending...
I20260619 20:35:45.892431 24143 ts_manager.cc:194] Registered new tserver with Master: eac21ab86d3a4abbbb5c1ed5e88d2cc8 (127.4.83.66:33361)
I20260619 20:35:45.892778 24143 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.83.66:36775
I20260619 20:35:45.893509 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 24335
I20260619 20:35:45.893606 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-1/wal/instance
I20260619 20:35:45.893756 4429 external_mini_cluster.cc:1428] Setting key 2fbb1dea2897d2e513fafb3d042a4f72
I20260619 20:35:45.894881 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.83.67:0
--local_ip_for_outbound_sockets=127.4.83.67
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:45309
--builtin_ntp_servers=127.4.83.84:44095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 20:35:45.976647 24466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:45.976892 24466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:45.976918 24466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:45.978156 24466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:45.978210 24466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:45.978237 24466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.83.67
I20260619 20:35:45.979676 24466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:44095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.83.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.83.67
--webserver_port=0
--tserver_master_addrs=127.4.83.126:45309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.83.67
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:45.979940 24466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:45.980266 24466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:45.985225 24471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:45.985225 24472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:45.985337 24466 server_base.cc:1034] running on GCE node
W20260619 20:35:45.985229 24474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:45.985653 24466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:45.985930 24466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:45.987111 24466 hybrid_clock.cc:648] HybridClock initialized: now 1781901345987097 us; error 31 us; skew 500 ppm
I20260619 20:35:45.988695 24466 webserver.cc:460] Webserver started at http://127.4.83.67:44165/ using document root <none> and password file <none>
I20260619 20:35:45.988958 24466 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:45.989015 24466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:45.989166 24466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:45.990535 24466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-2/data/instance:
uuid: "499f937cb4c743a0b5def76a777f0053"
format_stamp: "Formatted at 2026-06-19 20:35:45 on dist-test-slave-4m58"
server_key: "264b4bdef5bf80d62548af96c9c3e43d"
server_key_iv: "c59537a8b7f1cf7db0d2e1b8cbb6b037"
server_key_version: "encryptionkey@0"
I20260619 20:35:45.990866 24466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance:
uuid: "499f937cb4c743a0b5def76a777f0053"
format_stamp: "Formatted at 2026-06-19 20:35:45 on dist-test-slave-4m58"
server_key: "264b4bdef5bf80d62548af96c9c3e43d"
server_key_iv: "c59537a8b7f1cf7db0d2e1b8cbb6b037"
server_key_version: "encryptionkey@0"
I20260619 20:35:45.992151 24466 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:35:45.992858 24480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:45.993024 24466 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:35:45.993104 24466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-2/wal
uuid: "499f937cb4c743a0b5def76a777f0053"
format_stamp: "Formatted at 2026-06-19 20:35:45 on dist-test-slave-4m58"
server_key: "264b4bdef5bf80d62548af96c9c3e43d"
server_key_iv: "c59537a8b7f1cf7db0d2e1b8cbb6b037"
server_key_version: "encryptionkey@0"
I20260619 20:35:45.993167 24466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:46.005247 24466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:46.005566 24466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:46.005693 24466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:46.005903 24466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:35:46.006207 24466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:35:46.006239 24466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:46.006269 24466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:35:46.006292 24466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:46.012095 24466 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.67:46699
I20260619 20:35:46.012207 24593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.67:46699 every 8 connection(s)
I20260619 20:35:46.012506 24466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-2/data/info.pb
I20260619 20:35:46.017964 24184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:46.013224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.67:39505 (local address 127.4.83.126:45309)
0619 20:35:46.013271 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:46.013273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:46.013598 (+ 325us) server_negotiation.cc:407] Connection header received
0619 20:35:46.014097 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:46.014099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:46.014110 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:46.014168 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:46.014973 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:46.015488 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:46.016555 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:46.016748 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:46.016812 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:46.017686 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:46.017688 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:46.017688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:35:46.017693 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:46.017709 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:35:46.017713 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:46.017757 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:46.017872 (+ 115us) server_negotiation.cc:299] Negotiation successful
0619 20:35:46.017897 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:35:46.018216 24596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:46.013181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:45309 (local address 127.4.83.67:39505)
0619 20:35:46.013444 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:46.013515 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:46.013992 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:46.014240 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:46.014244 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:46.014252 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:35:46.014851 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:46.014857 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:46.015588 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:46.015589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:46.016459 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:46.016462 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:46.016851 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:46.016852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:46.016980 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:46.017544 (+ 564us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:35:46.017544 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:35:46.017574 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:35:46.017600 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:35:46.017792 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:35:46.017799 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:35:46.017802 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:35:46.017873 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 20:35:46.017966 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":107,"threads_started":1}
I20260619 20:35:46.018685 24594 heartbeater.cc:347] Connected to a master server at 127.4.83.126:45309
I20260619 20:35:46.018777 24594 heartbeater.cc:464] Registering TS with master...
I20260619 20:35:46.018950 24594 heartbeater.cc:511] Master 127.4.83.126:45309 requested a full tablet report, sending...
I20260619 20:35:46.019346 24143 ts_manager.cc:194] Registered new tserver with Master: 499f937cb4c743a0b5def76a777f0053 (127.4.83.67:46699)
I20260619 20:35:46.019692 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 24466
I20260619 20:35:46.019733 24143 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.83.67:39505
I20260619 20:35:46.019781 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-2/wal/instance
I20260619 20:35:46.020036 4429 external_mini_cluster.cc:1428] Setting key 0c6161f4df95aafc0f6285bce3e9ce17
I20260619 20:35:46.020845 4429 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:35:46.024999 24184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:46.022156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59314 (local address 127.4.83.126:45309)
0619 20:35:46.022219 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:46.022221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:46.022309 (+ 88us) server_negotiation.cc:407] Connection header received
0619 20:35:46.022372 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:46.022373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:46.022391 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:46.022436 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:46.022574 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:46.023056 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:46.023795 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:46.024049 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:46.024104 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:46.024769 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:46.024771 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:46.024771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:35:46.024777 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:46.024792 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:35:46.024796 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:46.024840 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:46.024887 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 20:35:46.024915 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 20:35:46.026767 24143 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59314:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:35:46.028393 24143 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:35:46.034896 24605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:46.030747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39030 (local address 127.4.83.65:41119)
0619 20:35:46.030932 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:46.030937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:46.031209 (+ 272us) server_negotiation.cc:407] Connection header received
0619 20:35:46.031253 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:46.031256 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:46.031270 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:46.031334 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:46.031535 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:46.032344 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:46.033716 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:46.034476 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:46.034550 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:46.034650 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:46.034756 (+ 106us) server_negotiation.cc:299] Negotiation successful
0619 20:35:46.034802 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
I20260619 20:35:46.034904 24603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:46.030575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:41119 (local address 127.0.0.1:39030)
0619 20:35:46.031129 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:46.031138 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:46.031184 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:46.031336 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:46.031337 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:46.031339 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:46.031458 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:46.031461 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:46.032419 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:46.032420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:46.033644 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:46.033646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:46.034554 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:46.034555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:46.034644 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:46.034660 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 20:35:46.034739 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 20:35:46.034801 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":21,"thread_start_us":106,"threads_started":1}
I20260619 20:35:46.035259 24606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:46.030822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58858 (local address 127.4.83.67:46699)
0619 20:35:46.030987 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:46.030991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:46.031085 (+ 94us) server_negotiation.cc:407] Connection header received
0619 20:35:46.031136 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:46.031138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:46.031153 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:46.031236 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:46.031483 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:46.032698 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:46.033893 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:46.034702 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:46.034763 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:46.034836 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:46.035163 (+ 327us) server_negotiation.cc:299] Negotiation successful
0619 20:35:46.035199 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":77,"threads_started":1}
I20260619 20:35:46.035108 24604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:46.030575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:46699 (local address 127.0.0.1:58858)
0619 20:35:46.031014 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:46.031022 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:46.031070 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:46.031322 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:46.031322 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:46.031326 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:46.031403 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:46.031405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:46.032784 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:46.032785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:46.033821 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:46.033822 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:46.034871 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:46.034872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:46.034949 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:46.034953 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:35:46.035001 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 20:35:46.035047 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"mutex_wait_us":3,"thread_start_us":63,"threads_started":1}
I20260619 20:35:46.035953 24602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:46.030575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:33361 (local address 127.0.0.1:60450)
0619 20:35:46.030780 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:46.030826 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:46.030902 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:46.031319 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:46.031322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:46.031325 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:46.031403 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:46.031406 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:46.033889 (+ 2483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:46.033890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:46.034685 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:46.034688 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:46.035702 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:46.035704 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:46.035767 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:46.035771 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:35:46.035817 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 20:35:46.035882 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":3,"thread_start_us":107,"threads_started":1}
I20260619 20:35:46.036248 24607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:46.030959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60450 (local address 127.4.83.66:33361)
0619 20:35:46.031110 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:46.031114 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:46.031122 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:35:46.031155 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:46.031156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:46.031168 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:46.031236 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:46.031483 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:46.032701 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:46.034780 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:46.035519 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:46.035584 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:46.035664 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:46.036122 (+ 458us) server_negotiation.cc:299] Negotiation successful
0619 20:35:46.036166 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
I20260619 20:35:46.036258 24528 tablet_service.cc:1467] Processing CreateTablet for tablet d8b9060b5a6348fca0100ccc8bffe5c6 (DEFAULT_TABLE table=test-table [id=d40c8e75c82a4c46bc992f945719541f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:35:46.036299 24266 tablet_service.cc:1467] Processing CreateTablet for tablet d8b9060b5a6348fca0100ccc8bffe5c6 (DEFAULT_TABLE table=test-table [id=d40c8e75c82a4c46bc992f945719541f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:35:46.036615 24528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8b9060b5a6348fca0100ccc8bffe5c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:46.036623 24266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8b9060b5a6348fca0100ccc8bffe5c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:46.036882 24397 tablet_service.cc:1467] Processing CreateTablet for tablet d8b9060b5a6348fca0100ccc8bffe5c6 (DEFAULT_TABLE table=test-table [id=d40c8e75c82a4c46bc992f945719541f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:35:46.037168 24397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8b9060b5a6348fca0100ccc8bffe5c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:46.039182 24608 tablet_bootstrap.cc:492] T d8b9060b5a6348fca0100ccc8bffe5c6 P 499f937cb4c743a0b5def76a777f0053: Bootstrap starting.
I20260619 20:35:46.039896 24608 tablet_bootstrap.cc:654] T d8b9060b5a6348fca0100ccc8bffe5c6 P 499f937cb4c743a0b5def76a777f0053: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:46.040190 24608 log.cc:824] T d8b9060b5a6348fca0100ccc8bffe5c6 P 499f937cb4c743a0b5def76a777f0053: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:46.040510 24610 tablet_bootstrap.cc:492] T d8b9060b5a6348fca0100ccc8bffe5c6 P 746524891023432999cee081e25ed660: Bootstrap starting.
I20260619 20:35:46.040601 24609 tablet_bootstrap.cc:492] T d8b9060b5a6348fca0100ccc8bffe5c6 P eac21ab86d3a4abbbb5c1ed5e88d2cc8: Bootstrap starting.
I20260619 20:35:46.041057 24608 tablet_bootstrap.cc:492] T d8b9060b5a6348fca0100ccc8bffe5c6 P 499f937cb4c743a0b5def76a777f0053: No bootstrap required, opened a new log
I20260619 20:35:46.041124 24608 ts_tablet_manager.cc:1397] T d8b9060b5a6348fca0100ccc8bffe5c6 P 499f937cb4c743a0b5def76a777f0053: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:35:46.041194 24610 tablet_bootstrap.cc:654] T d8b9060b5a6348fca0100ccc8bffe5c6 P 746524891023432999cee081e25ed660: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:46.041194 24609 tablet_bootstrap.cc:654] T d8b9060b5a6348fca0100ccc8bffe5c6 P eac21ab86d3a4abbbb5c1ed5e88d2cc8: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:46.041514 24609 log.cc:824] T d8b9060b5a6348fca0100ccc8bffe5c6 P eac21ab86d3a4abbbb5c1ed5e88d2cc8: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:46.041514 24610 log.cc:824] T d8b9060b5a6348fca0100ccc8bffe5c6 P 746524891023432999cee081e25ed660: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:46.042318 24610 tablet_bootstrap.cc:492] T d8b9060b5a6348fca0100ccc8bffe5c6 P 746524891023432999cee081e25ed660: No bootstrap required, opened a new log
I20260619 20:35:46.042392 24610 ts_tablet_manager.cc:1397] T d8b9060b5a6348fca0100ccc8bffe5c6 P 746524891023432999cee081e25ed660: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:35:46.042583 24609 tablet_bootstrap.cc:492] T d8b9060b5a6348fca0100ccc8bffe5c6 P eac21ab86d3a4abbbb5c1ed5e88d2cc8: No bootstrap required, opened a new log
I20260619 20:35:46.042654 24609 ts_tablet_manager.cc:1397] T d8b9060b5a6348fca0100ccc8bffe5c6 P eac21ab86d3a4abbbb5c1ed5e88d2cc8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:35:46.043676 24608 raft_consensus.cc:348] T d8b9060b5a6348fca0100ccc8bffe5c6 P 499f937cb4c743a0b5def76a777f0053 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac21ab86d3a4abbbb5c1ed5e88d2cc8" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33361 } } peers { permanent_uuid: "746524891023432999cee081e25ed660" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41119 } } peers { permanent_uuid: "499f937cb4c743a0b5def76a777f0053" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 46699 } }
I20260619 20:35:46.043884 24608 raft_consensus.cc:374] T d8b9060b5a6348fca0100ccc8bffe5c6 P 499f937cb4c743a0b5def76a777f0053 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:46.043923 24608 raft_consensus.cc:729] T d8b9060b5a6348fca0100ccc8bffe5c6 P 499f937cb4c743a0b5def76a777f0053 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 499f937cb4c743a0b5def76a777f0053, State: Initialized, Role: FOLLOWER
I20260619 20:35:46.044071 24608 consensus_queue.cc:260] T d8b9060b5a6348fca0100ccc8bffe5c6 P 499f937cb4c743a0b5def76a777f0053 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac21ab86d3a4abbbb5c1ed5e88d2cc8" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33361 } } peers { permanent_uuid: "746524891023432999cee081e25ed660" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41119 } } peers { permanent_uuid: "499f937cb4c743a0b5def76a777f0053" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 46699 } }
I20260619 20:35:46.044363 24608 ts_tablet_manager.cc:1428] T d8b9060b5a6348fca0100ccc8bffe5c6 P 499f937cb4c743a0b5def76a777f0053: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:35:46.044366 24610 raft_consensus.cc:348] T d8b9060b5a6348fca0100ccc8bffe5c6 P 746524891023432999cee081e25ed660 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac21ab86d3a4abbbb5c1ed5e88d2cc8" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33361 } } peers { permanent_uuid: "746524891023432999cee081e25ed660" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41119 } } peers { permanent_uuid: "499f937cb4c743a0b5def76a777f0053" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 46699 } }
I20260619 20:35:46.044481 24594 heartbeater.cc:503] Master 127.4.83.126:45309 was elected leader, sending a full tablet report...
I20260619 20:35:46.044508 24610 raft_consensus.cc:374] T d8b9060b5a6348fca0100ccc8bffe5c6 P 746524891023432999cee081e25ed660 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:46.044546 24610 raft_consensus.cc:729] T d8b9060b5a6348fca0100ccc8bffe5c6 P 746524891023432999cee081e25ed660 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 746524891023432999cee081e25ed660, State: Initialized, Role: FOLLOWER
I20260619 20:35:46.044651 24610 consensus_queue.cc:260] T d8b9060b5a6348fca0100ccc8bffe5c6 P 746524891023432999cee081e25ed660 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac21ab86d3a4abbbb5c1ed5e88d2cc8" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33361 } } peers { permanent_uuid: "746524891023432999cee081e25ed660" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41119 } } peers { permanent_uuid: "499f937cb4c743a0b5def76a777f0053" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 46699 } }
I20260619 20:35:46.044880 24610 ts_tablet_manager.cc:1428] T d8b9060b5a6348fca0100ccc8bffe5c6 P 746524891023432999cee081e25ed660: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 20:35:46.044951 24332 heartbeater.cc:503] Master 127.4.83.126:45309 was elected leader, sending a full tablet report...
I20260619 20:35:46.045575 24609 raft_consensus.cc:348] T d8b9060b5a6348fca0100ccc8bffe5c6 P eac21ab86d3a4abbbb5c1ed5e88d2cc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac21ab86d3a4abbbb5c1ed5e88d2cc8" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33361 } } peers { permanent_uuid: "746524891023432999cee081e25ed660" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41119 } } peers { permanent_uuid: "499f937cb4c743a0b5def76a777f0053" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 46699 } }
I20260619 20:35:46.045734 24609 raft_consensus.cc:374] T d8b9060b5a6348fca0100ccc8bffe5c6 P eac21ab86d3a4abbbb5c1ed5e88d2cc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:46.045785 24609 raft_consensus.cc:729] T d8b9060b5a6348fca0100ccc8bffe5c6 P eac21ab86d3a4abbbb5c1ed5e88d2cc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac21ab86d3a4abbbb5c1ed5e88d2cc8, State: Initialized, Role: FOLLOWER
I20260619 20:35:46.045899 24609 consensus_queue.cc:260] T d8b9060b5a6348fca0100ccc8bffe5c6 P eac21ab86d3a4abbbb5c1ed5e88d2cc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac21ab86d3a4abbbb5c1ed5e88d2cc8" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33361 } } peers { permanent_uuid: "746524891023432999cee081e25ed660" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41119 } } peers { permanent_uuid: "499f937cb4c743a0b5def76a777f0053" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 46699 } }
I20260619 20:35:46.046108 24609 ts_tablet_manager.cc:1428] T d8b9060b5a6348fca0100ccc8bffe5c6 P eac21ab86d3a4abbbb5c1ed5e88d2cc8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:35:46.046168 24463 heartbeater.cc:503] Master 127.4.83.126:45309 was elected leader, sending a full tablet report...
W20260619 20:35:46.136709 24464 tablet.cc:2369] T d8b9060b5a6348fca0100ccc8bffe5c6 P eac21ab86d3a4abbbb5c1ed5e88d2cc8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:35:46.234349 24615 raft_consensus.cc:482] T d8b9060b5a6348fca0100ccc8bffe5c6 P 746524891023432999cee081e25ed660 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:35:46.234478 24615 raft_consensus.cc:504] T d8b9060b5a6348fca0100ccc8bffe5c6 P 746524891023432999cee081e25ed660 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac21ab86d3a4abbbb5c1ed5e88d2cc8" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33361 } } peers { permanent_uuid: "746524891023432999cee081e25ed660" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41119 } } peers { permanent_uuid: "499f937cb4c743a0b5def76a777f0053" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 46699 } }
I20260619 20:35:46.234789 24615 leader_election.cc:288] T d8b9060b5a6348fca0100ccc8bffe5c6 P 746524891023432999cee081e25ed660 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac21ab86d3a4abbbb5c1ed5e88d2cc8 (127.4.83.66:33361), 499f937cb4c743a0b5def76a777f0053 (127.4.83.67:46699)
I20260619 20:35:46.237607 24607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:46.235060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:50071 (local address 127.4.83.66:33361)
0619 20:35:46.235108 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:46.235110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:46.235214 (+ 104us) server_negotiation.cc:407] Connection header received
0619 20:35:46.235274 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:46.235275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:46.235283 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:46.235338 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:46.235496 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:46.235969 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:46.236958 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:46.237306 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:46.237371 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:46.237380 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:46.237495 (+ 115us) server_negotiation.cc:299] Negotiation successful
0619 20:35:46.237529 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:35:46.237605 24606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:46.235057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.83.65:45217 (local address 127.4.83.67:46699)
0619 20:35:46.235101 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:46.235103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:46.235115 (+ 12us) server_negotiation.cc:407] Connection header received
0619 20:35:46.235175 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:46.235176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:46.235184 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:46.235242 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:35:46.235417 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:46.235952 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:46.236941 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:46.237284 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:46.237343 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:46.237353 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:46.237496 (+ 143us) server_negotiation.cc:299] Negotiation successful
0619 20:35:46.237533 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:35:46.237620 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:46.234980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.67:46699 (local address 127.4.83.65:45217)
0619 20:35:46.235045 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:46.235056 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:46.235101 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:46.235258 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:46.235259 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:46.235260 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:46.235343 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:46.235345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:46.236032 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:46.236033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:46.236855 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:46.236857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:46.237352 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:46.237353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:46.237433 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:46.237437 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:35:46.237483 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 20:35:46.237529 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":41}
I20260619 20:35:46.237628 24617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:46.234980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.66:33361 (local address 127.4.83.65:50071)
0619 20:35:46.235128 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:46.235138 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:35:46.235217 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:35:46.235351 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:35:46.235351 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:35:46.235352 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:35:46.235430 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:46.235432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:46.236047 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:46.236048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:46.236851 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:35:46.236853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:35:46.237381 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:35:46.237382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:35:46.237434 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:46.237437 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:35:46.237483 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 20:35:46.237523 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":33,"thread_start_us":63,"threads_started":1}
I20260619 20:35:46.237982 24417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8b9060b5a6348fca0100ccc8bffe5c6" candidate_uuid: "746524891023432999cee081e25ed660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac21ab86d3a4abbbb5c1ed5e88d2cc8" is_pre_election: true
I20260619 20:35:46.237983 24548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8b9060b5a6348fca0100ccc8bffe5c6" candidate_uuid: "746524891023432999cee081e25ed660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "499f937cb4c743a0b5def76a777f0053" is_pre_election: true
I20260619 20:35:46.238147 24548 raft_consensus.cc:2454] T d8b9060b5a6348fca0100ccc8bffe5c6 P 499f937cb4c743a0b5def76a777f0053 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 746524891023432999cee081e25ed660 in term 0.
I20260619 20:35:46.238147 24417 raft_consensus.cc:2454] T d8b9060b5a6348fca0100ccc8bffe5c6 P eac21ab86d3a4abbbb5c1ed5e88d2cc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 746524891023432999cee081e25ed660 in term 0.
I20260619 20:35:46.238368 24222 leader_election.cc:302] T d8b9060b5a6348fca0100ccc8bffe5c6 P 746524891023432999cee081e25ed660 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 746524891023432999cee081e25ed660, eac21ab86d3a4abbbb5c1ed5e88d2cc8; no voters:
I20260619 20:35:46.238538 24615 raft_consensus.cc:2781] T d8b9060b5a6348fca0100ccc8bffe5c6 P 746524891023432999cee081e25ed660 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:35:46.238608 24615 raft_consensus.cc:482] T d8b9060b5a6348fca0100ccc8bffe5c6 P 746524891023432999cee081e25ed660 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:35:46.238642 24615 raft_consensus.cc:3037] T d8b9060b5a6348fca0100ccc8bffe5c6 P 746524891023432999cee081e25ed660 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:46.239467 24615 raft_consensus.cc:504] T d8b9060b5a6348fca0100ccc8bffe5c6 P 746524891023432999cee081e25ed660 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac21ab86d3a4abbbb5c1ed5e88d2cc8" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33361 } } peers { permanent_uuid: "746524891023432999cee081e25ed660" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41119 } } peers { permanent_uuid: "499f937cb4c743a0b5def76a777f0053" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 46699 } }
I20260619 20:35:46.239619 24615 leader_election.cc:288] T d8b9060b5a6348fca0100ccc8bffe5c6 P 746524891023432999cee081e25ed660 [CANDIDATE]: Term 1 election: Requested vote from peers eac21ab86d3a4abbbb5c1ed5e88d2cc8 (127.4.83.66:33361), 499f937cb4c743a0b5def76a777f0053 (127.4.83.67:46699)
I20260619 20:35:46.239853 24548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8b9060b5a6348fca0100ccc8bffe5c6" candidate_uuid: "746524891023432999cee081e25ed660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "499f937cb4c743a0b5def76a777f0053"
I20260619 20:35:46.239856 24417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8b9060b5a6348fca0100ccc8bffe5c6" candidate_uuid: "746524891023432999cee081e25ed660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac21ab86d3a4abbbb5c1ed5e88d2cc8"
I20260619 20:35:46.239940 24417 raft_consensus.cc:3037] T d8b9060b5a6348fca0100ccc8bffe5c6 P eac21ab86d3a4abbbb5c1ed5e88d2cc8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:46.239938 24548 raft_consensus.cc:3037] T d8b9060b5a6348fca0100ccc8bffe5c6 P 499f937cb4c743a0b5def76a777f0053 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:46.240613 24548 raft_consensus.cc:2454] T d8b9060b5a6348fca0100ccc8bffe5c6 P 499f937cb4c743a0b5def76a777f0053 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 746524891023432999cee081e25ed660 in term 1.
I20260619 20:35:46.240661 24417 raft_consensus.cc:2454] T d8b9060b5a6348fca0100ccc8bffe5c6 P eac21ab86d3a4abbbb5c1ed5e88d2cc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 746524891023432999cee081e25ed660 in term 1.
I20260619 20:35:46.240792 24219 leader_election.cc:302] T d8b9060b5a6348fca0100ccc8bffe5c6 P 746524891023432999cee081e25ed660 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 499f937cb4c743a0b5def76a777f0053, 746524891023432999cee081e25ed660; no voters:
I20260619 20:35:46.240926 24615 raft_consensus.cc:2781] T d8b9060b5a6348fca0100ccc8bffe5c6 P 746524891023432999cee081e25ed660 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:35:46.241056 24615 raft_consensus.cc:686] T d8b9060b5a6348fca0100ccc8bffe5c6 P 746524891023432999cee081e25ed660 [term 1 LEADER]: Becoming Leader. State: Replica: 746524891023432999cee081e25ed660, State: Running, Role: LEADER
I20260619 20:35:46.241145 24615 consensus_queue.cc:237] T d8b9060b5a6348fca0100ccc8bffe5c6 P 746524891023432999cee081e25ed660 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac21ab86d3a4abbbb5c1ed5e88d2cc8" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33361 } } peers { permanent_uuid: "746524891023432999cee081e25ed660" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41119 } } peers { permanent_uuid: "499f937cb4c743a0b5def76a777f0053" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 46699 } }
I20260619 20:35:46.242028 24143 catalog_manager.cc:5495] T d8b9060b5a6348fca0100ccc8bffe5c6 P 746524891023432999cee081e25ed660 reported cstate change: term changed from 0 to 1, leader changed from <none> to 746524891023432999cee081e25ed660 (127.4.83.65). New cstate: current_term: 1 leader_uuid: "746524891023432999cee081e25ed660" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac21ab86d3a4abbbb5c1ed5e88d2cc8" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "746524891023432999cee081e25ed660" member_type: VOTER last_known_addr { host: "127.4.83.65" port: 41119 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "499f937cb4c743a0b5def76a777f0053" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 46699 } health_report { overall_health: UNKNOWN } } }
W20260619 20:35:46.251045 24333 tablet.cc:2369] T d8b9060b5a6348fca0100ccc8bffe5c6 P 746524891023432999cee081e25ed660: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:35:46.262266 24605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:46.259992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39036 (local address 127.4.83.65:41119)
0619 20:35:46.260045 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:46.260047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:46.260057 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:35:46.260094 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:46.260095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:46.260113 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:46.260174 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:35:46.260291 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:46.260758 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:46.261557 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:46.261758 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:46.261851 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:46.261981 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:35:46.262086 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:35:46.262144 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:46.262161 (+ 17us) server_negotiation.cc:299] Negotiation successful
0619 20:35:46.262181 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260619 20:35:46.263032 24595 tablet.cc:2369] T d8b9060b5a6348fca0100ccc8bffe5c6 P 499f937cb4c743a0b5def76a777f0053: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:35:46.264225 24417 raft_consensus.cc:1261] T d8b9060b5a6348fca0100ccc8bffe5c6 P eac21ab86d3a4abbbb5c1ed5e88d2cc8 [term 1 FOLLOWER]: Refusing update from remote peer 746524891023432999cee081e25ed660: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:35:46.264225 24548 raft_consensus.cc:1261] T d8b9060b5a6348fca0100ccc8bffe5c6 P 499f937cb4c743a0b5def76a777f0053 [term 1 FOLLOWER]: Refusing update from remote peer 746524891023432999cee081e25ed660: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:35:46.264612 24615 consensus_queue.cc:1035] T d8b9060b5a6348fca0100ccc8bffe5c6 P 746524891023432999cee081e25ed660 [LEADER]: Connected to new peer: Peer: permanent_uuid: "499f937cb4c743a0b5def76a777f0053" member_type: VOTER last_known_addr { host: "127.4.83.67" port: 46699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:35:46.264757 24618 consensus_queue.cc:1035] T d8b9060b5a6348fca0100ccc8bffe5c6 P 746524891023432999cee081e25ed660 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac21ab86d3a4abbbb5c1ed5e88d2cc8" member_type: VOTER last_known_addr { host: "127.4.83.66" port: 33361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:35:46.266999 24621 mvcc.cc:205] Tried to move back new op lower bound from 7298667914295443456 to 7298667914204332032. Current Snapshot: MvccSnapshot[applied={T|T < 7298667914295443456}]
I20260619 20:35:46.267227 24623 mvcc.cc:205] Tried to move back new op lower bound from 7298667914295443456 to 7298667914204332032. Current Snapshot: MvccSnapshot[applied={T|T < 7298667914295443456}]
I20260619 20:35:46.267266 24622 mvcc.cc:205] Tried to move back new op lower bound from 7298667914295443456 to 7298667914204332032. Current Snapshot: MvccSnapshot[applied={T|T < 7298667914295443456}]
I20260619 20:35:46.269562 24144 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:35:46.269668 24144 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:35:46.270457 24144 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7201b746b8fa4816b95c6dc1d9c4c0f4: Sending DeleteTablet for 3 replicas of tablet d8b9060b5a6348fca0100ccc8bffe5c6
I20260619 20:35:46.270867 24266 tablet_service.cc:1514] Processing DeleteTablet for tablet d8b9060b5a6348fca0100ccc8bffe5c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:35:46 UTC) from {username='slave'} at 127.0.0.1:39030
I20260619 20:35:46.270870 24528 tablet_service.cc:1514] Processing DeleteTablet for tablet d8b9060b5a6348fca0100ccc8bffe5c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:35:46 UTC) from {username='slave'} at 127.0.0.1:58858
I20260619 20:35:46.271202 24632 tablet_replica.cc:330] T d8b9060b5a6348fca0100ccc8bffe5c6 P 746524891023432999cee081e25ed660: stopping tablet replica
I20260619 20:35:46.271220 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 24204
I20260619 20:35:46.271242 24397 tablet_service.cc:1514] Processing DeleteTablet for tablet d8b9060b5a6348fca0100ccc8bffe5c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:35:46 UTC) from {username='slave'} at 127.0.0.1:60450
I20260619 20:35:46.272917 24633 tablet_replica.cc:330] T d8b9060b5a6348fca0100ccc8bffe5c6 P eac21ab86d3a4abbbb5c1ed5e88d2cc8: stopping tablet replica
I20260619 20:35:46.273046 24633 raft_consensus.cc:2229] T d8b9060b5a6348fca0100ccc8bffe5c6 P eac21ab86d3a4abbbb5c1ed5e88d2cc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:35:46.273264 24633 raft_consensus.cc:2258] T d8b9060b5a6348fca0100ccc8bffe5c6 P eac21ab86d3a4abbbb5c1ed5e88d2cc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:35:46.273509 24631 tablet_replica.cc:330] T d8b9060b5a6348fca0100ccc8bffe5c6 P 499f937cb4c743a0b5def76a777f0053: stopping tablet replica
I20260619 20:35:46.273617 24631 raft_consensus.cc:2229] T d8b9060b5a6348fca0100ccc8bffe5c6 P 499f937cb4c743a0b5def76a777f0053 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:35:46.273710 24631 raft_consensus.cc:2258] T d8b9060b5a6348fca0100ccc8bffe5c6 P 499f937cb4c743a0b5def76a777f0053 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:35:46.273739 24633 ts_tablet_manager.cc:1905] T d8b9060b5a6348fca0100ccc8bffe5c6 P eac21ab86d3a4abbbb5c1ed5e88d2cc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:35:46.274125 24631 ts_tablet_manager.cc:1905] T d8b9060b5a6348fca0100ccc8bffe5c6 P 499f937cb4c743a0b5def76a777f0053: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:35:46.275561 24631 ts_tablet_manager.cc:1918] T d8b9060b5a6348fca0100ccc8bffe5c6 P 499f937cb4c743a0b5def76a777f0053: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260619 20:35:46.275553 24129 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.83.65:41119 (error 108)
I20260619 20:35:46.275641 24631 log.cc:1196] T d8b9060b5a6348fca0100ccc8bffe5c6 P 499f937cb4c743a0b5def76a777f0053: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-2/wal/wals/d8b9060b5a6348fca0100ccc8bffe5c6
I20260619 20:35:46.275672 24633 ts_tablet_manager.cc:1918] T d8b9060b5a6348fca0100ccc8bffe5c6 P eac21ab86d3a4abbbb5c1ed5e88d2cc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:35:46.275744 24633 log.cc:1196] T d8b9060b5a6348fca0100ccc8bffe5c6 P eac21ab86d3a4abbbb5c1ed5e88d2cc8: Deleting WAL directory at /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901102012824-4429-0/minicluster-data/ts-1/wal/wals/d8b9060b5a6348fca0100ccc8bffe5c6
I20260619 20:35:46.276070 24631 ts_tablet_manager.cc:1939] T d8b9060b5a6348fca0100ccc8bffe5c6 P 499f937cb4c743a0b5def76a777f0053: Deleting consensus metadata
I20260619 20:35:46.276161 24633 ts_tablet_manager.cc:1939] T d8b9060b5a6348fca0100ccc8bffe5c6 P eac21ab86d3a4abbbb5c1ed5e88d2cc8: Deleting consensus metadata
I20260619 20:35:46.276479 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 24335
I20260619 20:35:46.276501 24602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:46.276333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.65:41119 (local address 127.0.0.1:39050)
0619 20:35:46.276402 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 20:35:46.276461 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.65:41119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 20:35:46.276556 24128 catalog_manager.cc:4841] TS 499f937cb4c743a0b5def76a777f0053 (127.4.83.67:46699): tablet d8b9060b5a6348fca0100ccc8bffe5c6 (table test-table [id=d40c8e75c82a4c46bc992f945719541f]) successfully deleted
I20260619 20:35:46.276582 24131 catalog_manager.cc:4841] TS eac21ab86d3a4abbbb5c1ed5e88d2cc8 (127.4.83.66:33361): tablet d8b9060b5a6348fca0100ccc8bffe5c6 (table test-table [id=d40c8e75c82a4c46bc992f945719541f]) successfully deleted
W20260619 20:35:46.276626 24129 catalog_manager.cc:4584] TS 746524891023432999cee081e25ed660 (127.4.83.65:41119): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d8b9060b5a6348fca0100ccc8bffe5c6: Network error: Client connection negotiation failed: client connection to 127.4.83.65:41119: connect: Connection refused (error 111)
I20260619 20:35:46.280699 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 24466
I20260619 20:35:46.285254 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 24113
2026-06-19T20:35:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (817 ms)
[----------] 6 tests from EncryptionPolicyTest (7933 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:35:46 dist-test-slave-4m58 krb5kdc[24637](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:35:46 dist-test-slave-4m58 krb5kdc[24637](info): set up 2 sockets
Jun 19 20:35:46 dist-test-slave-4m58 krb5kdc[24637](info): commencing operation
krb5kdc: starting...
W20260619 20:35:47.249224 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.946s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:35:47 dist-test-slave-4m58 krb5kdc[24637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:35:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:35:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:35:47.395907 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37335
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37335
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:35:47.493645 24650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:47.493952 24650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:47.493980 24650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:47.495409 24650 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:35:47.495466 24650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:47.495481 24650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:47.495493 24650 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:35:47.495503 24650 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:35:47.497365 24650 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37335
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:47.497653 24650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:47.497885 24650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:47.503463 24656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:47.503463 24655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:47.503463 24658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:47.503944 24650 server_base.cc:1034] running on GCE node
I20260619 20:35:47.504256 24650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:47.504616 24650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:47.505851 24650 hybrid_clock.cc:648] HybridClock initialized: now 1781901347505832 us; error 44 us; skew 500 ppm
Jun 19 20:35:47 dist-test-slave-4m58 krb5kdc[24637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901347, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:35:47.509162 24650 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:35:47.509850 24650 webserver.cc:460] Webserver started at http://127.0.0.1:45185/ using document root <none> and password file <none>
I20260619 20:35:47.510145 24650 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:47.510207 24650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:47.510367 24650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:47.511720 24650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "9cbaa028dfb64be696901e0bd496c3a5"
format_stamp: "Formatted at 2026-06-19 20:35:47 on dist-test-slave-4m58"
server_key: "9255216d0ff05cfc87bedf3048fc02bc"
server_key_iv: "275553169843afc44ba8a0fabeef66a9"
server_key_version: "encryptionkey@0"
I20260619 20:35:47.512248 24650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "9cbaa028dfb64be696901e0bd496c3a5"
format_stamp: "Formatted at 2026-06-19 20:35:47 on dist-test-slave-4m58"
server_key: "9255216d0ff05cfc87bedf3048fc02bc"
server_key_iv: "275553169843afc44ba8a0fabeef66a9"
server_key_version: "encryptionkey@0"
I20260619 20:35:47.514052 24650 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260619 20:35:47.514956 24665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:47.515153 24650 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:47.515254 24650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "9cbaa028dfb64be696901e0bd496c3a5"
format_stamp: "Formatted at 2026-06-19 20:35:47 on dist-test-slave-4m58"
server_key: "9255216d0ff05cfc87bedf3048fc02bc"
server_key_iv: "275553169843afc44ba8a0fabeef66a9"
server_key_version: "encryptionkey@0"
I20260619 20:35:47.515336 24650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:47.552637 24650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:47.555446 24650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:47.555639 24650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:47.560581 24650 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37335
I20260619 20:35:47.560622 24717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37335 every 8 connection(s)
I20260619 20:35:47.561080 24650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:35:47.562355 24718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:47.563792 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 24650
I20260619 20:35:47.564105 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:35:47.564411 4429 external_mini_cluster.cc:1428] Setting key b87f0b4725da76d6ad94f51a62d62896
I20260619 20:35:47.566550 24718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cbaa028dfb64be696901e0bd496c3a5: Bootstrap starting.
I20260619 20:35:47.567510 24718 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9cbaa028dfb64be696901e0bd496c3a5: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:47.567879 24718 log.cc:824] T 00000000000000000000000000000000 P 9cbaa028dfb64be696901e0bd496c3a5: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:47.568760 24718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cbaa028dfb64be696901e0bd496c3a5: No bootstrap required, opened a new log
I20260619 20:35:47.571523 24718 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9cbaa028dfb64be696901e0bd496c3a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cbaa028dfb64be696901e0bd496c3a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37335 } }
I20260619 20:35:47.571815 24718 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9cbaa028dfb64be696901e0bd496c3a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:47.571911 24718 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9cbaa028dfb64be696901e0bd496c3a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cbaa028dfb64be696901e0bd496c3a5, State: Initialized, Role: FOLLOWER
I20260619 20:35:47.572110 24718 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9cbaa028dfb64be696901e0bd496c3a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cbaa028dfb64be696901e0bd496c3a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37335 } }
I20260619 20:35:47.572212 24718 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9cbaa028dfb64be696901e0bd496c3a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:35:47.572259 24718 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9cbaa028dfb64be696901e0bd496c3a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:35:47.572315 24718 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9cbaa028dfb64be696901e0bd496c3a5 [term 0 FOLLOWER]: Advancing to term 1
Jun 19 20:35:47 dist-test-slave-4m58 krb5kdc[24637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 20:35:47.573133 24718 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9cbaa028dfb64be696901e0bd496c3a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cbaa028dfb64be696901e0bd496c3a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37335 } }
I20260619 20:35:47.573299 24718 leader_election.cc:302] T 00000000000000000000000000000000 P 9cbaa028dfb64be696901e0bd496c3a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9cbaa028dfb64be696901e0bd496c3a5; no voters:
I20260619 20:35:47.573570 24718 leader_election.cc:288] T 00000000000000000000000000000000 P 9cbaa028dfb64be696901e0bd496c3a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:35:47.573628 24723 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9cbaa028dfb64be696901e0bd496c3a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:35:47.573800 24723 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9cbaa028dfb64be696901e0bd496c3a5 [term 1 LEADER]: Becoming Leader. State: Replica: 9cbaa028dfb64be696901e0bd496c3a5, State: Running, Role: LEADER
I20260619 20:35:47.573940 24718 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9cbaa028dfb64be696901e0bd496c3a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:35:47.573985 24723 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9cbaa028dfb64be696901e0bd496c3a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cbaa028dfb64be696901e0bd496c3a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37335 } }
I20260619 20:35:47.574471 24723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cbaa028dfb64be696901e0bd496c3a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9cbaa028dfb64be696901e0bd496c3a5. Latest consensus state: current_term: 1 leader_uuid: "9cbaa028dfb64be696901e0bd496c3a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cbaa028dfb64be696901e0bd496c3a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37335 } } }
I20260619 20:35:47.574486 24725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cbaa028dfb64be696901e0bd496c3a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9cbaa028dfb64be696901e0bd496c3a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cbaa028dfb64be696901e0bd496c3a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37335 } } }
I20260619 20:35:47.574625 24725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cbaa028dfb64be696901e0bd496c3a5 [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:47.574616 24723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cbaa028dfb64be696901e0bd496c3a5 [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:47.574956 24731 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:35:47.575681 24731 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:35:47.577991 24731 catalog_manager.cc:1347] Generated new cluster ID: 23ad41f028f44f45b6f62b9f5d216548
I20260619 20:35:47.578064 24731 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:35:47.578752 24721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:47.565455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57967 (local address 127.0.0.1:37335)
0619 20:35:47.565667 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:47.565677 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:47.565707 (+ 30us) server_negotiation.cc:407] Connection header received
0619 20:35:47.566251 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:47.566260 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:47.566326 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:47.566487 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:47.567397 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:47.568684 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:47.569822 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:47.570329 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:47.570428 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:47.573630 (+ 3202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:47.573637 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:47.573638 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:35:47.573657 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:47.576062 (+ 2405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:47.576715 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:47.576719 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:47.576720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:47.576846 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:47.577130 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:47.577131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:47.577132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:47.577523 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:47.577731 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:47.578348 (+ 617us) server_negotiation.cc:299] Negotiation successful
0619 20:35:47.578480 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":88,"threads_started":1}
I20260619 20:35:47.581208 4429 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:35:47.593276 24721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:47.584073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47223 (local address 127.0.0.1:37335)
0619 20:35:47.584181 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:47.584184 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:47.584222 (+ 38us) server_negotiation.cc:407] Connection header received
0619 20:35:47.584256 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:47.584258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:47.584273 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:47.584336 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:47.585205 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:47.586044 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:47.587232 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:47.587625 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:47.587722 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:47.589569 (+ 1847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:47.589572 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:47.589572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:35:47.589585 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:47.591516 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:47.592225 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:47.592227 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:47.592228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:47.592319 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:47.592590 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:47.592591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:47.592592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:47.592776 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:47.592911 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:47.593050 (+ 139us) server_negotiation.cc:299] Negotiation successful
0619 20:35:47.593151 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 20:35:47.595460 24731 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:35:47.596413 24731 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:35:47.606810 24731 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9cbaa028dfb64be696901e0bd496c3a5: Generated new TSK 0
I20260619 20:35:47.607131 24731 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:35:47.609959 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 24650
W20260619 20:35:47.614687 24635 connection.cc:535] client connection to 127.0.0.1:37335 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37335): Connection reset by peer (error 104)
2026-06-19T20:35:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1324 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:35:47 dist-test-slave-4m58 krb5kdc[24749](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:35:47 dist-test-slave-4m58 krb5kdc[24749](info): set up 2 sockets
Jun 19 20:35:47 dist-test-slave-4m58 krb5kdc[24749](info): commencing operation
krb5kdc: starting...
W20260619 20:35:48.613098 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.984s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:35:48 dist-test-slave-4m58 krb5kdc[24749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:35:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:35:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:35:48.731555 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43633
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:35:48.818390 24762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:48.818738 24762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:48.818780 24762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:48.820750 24762 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:35:48.820873 24762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:48.820897 24762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:48.820917 24762 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:35:48.820935 24762 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:35:48.823276 24762 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43633
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:48.823666 24762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:48.823995 24762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:48.829699 24768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:48.829679 24767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:48.829679 24770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:48.830027 24762 server_base.cc:1034] running on GCE node
I20260619 20:35:48.830340 24762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:48.830693 24762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:48.831899 24762 hybrid_clock.cc:648] HybridClock initialized: now 1781901348831874 us; error 37 us; skew 500 ppm
Jun 19 20:35:48 dist-test-slave-4m58 krb5kdc[24749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901348, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:35:48.835208 24762 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:35:48.835819 24762 webserver.cc:460] Webserver started at http://127.0.0.1:36247/ using document root <none> and password file <none>
I20260619 20:35:48.836169 24762 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:48.836257 24762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:48.836414 24762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:48.837766 24762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "b0924321a9a54e29b5eb88c55af805dc"
format_stamp: "Formatted at 2026-06-19 20:35:48 on dist-test-slave-4m58"
server_key: "07df573096434e45e2f1393bcd6913de"
server_key_iv: "3aa3885390af3461a18de6f2c01b87b4"
server_key_version: "encryptionkey@0"
I20260619 20:35:48.838191 24762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "b0924321a9a54e29b5eb88c55af805dc"
format_stamp: "Formatted at 2026-06-19 20:35:48 on dist-test-slave-4m58"
server_key: "07df573096434e45e2f1393bcd6913de"
server_key_iv: "3aa3885390af3461a18de6f2c01b87b4"
server_key_version: "encryptionkey@0"
I20260619 20:35:48.839777 24762 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260619 20:35:48.840688 24777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:48.840986 24762 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:48.841104 24762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "b0924321a9a54e29b5eb88c55af805dc"
format_stamp: "Formatted at 2026-06-19 20:35:48 on dist-test-slave-4m58"
server_key: "07df573096434e45e2f1393bcd6913de"
server_key_iv: "3aa3885390af3461a18de6f2c01b87b4"
server_key_version: "encryptionkey@0"
I20260619 20:35:48.841185 24762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:48.851141 24762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:48.853590 24762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:48.853735 24762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:48.858604 24762 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43633
I20260619 20:35:48.858654 24829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43633 every 8 connection(s)
I20260619 20:35:48.859133 24762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:35:48.860375 24830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:48.863382 24830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0924321a9a54e29b5eb88c55af805dc: Bootstrap starting.
I20260619 20:35:48.864248 24830 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0924321a9a54e29b5eb88c55af805dc: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:48.864573 24830 log.cc:824] T 00000000000000000000000000000000 P b0924321a9a54e29b5eb88c55af805dc: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:48.865540 24830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0924321a9a54e29b5eb88c55af805dc: No bootstrap required, opened a new log
I20260619 20:35:48.866940 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 24762
I20260619 20:35:48.867046 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:35:48.867233 4429 external_mini_cluster.cc:1428] Setting key 2df57d1abc69646fc8db1311e74339f4
I20260619 20:35:48.867534 24830 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0924321a9a54e29b5eb88c55af805dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0924321a9a54e29b5eb88c55af805dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43633 } }
I20260619 20:35:48.867758 24830 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0924321a9a54e29b5eb88c55af805dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:48.867822 24830 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0924321a9a54e29b5eb88c55af805dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0924321a9a54e29b5eb88c55af805dc, State: Initialized, Role: FOLLOWER
I20260619 20:35:48.868050 24830 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0924321a9a54e29b5eb88c55af805dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0924321a9a54e29b5eb88c55af805dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43633 } }
I20260619 20:35:48.868155 24830 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0924321a9a54e29b5eb88c55af805dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:35:48.868201 24830 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0924321a9a54e29b5eb88c55af805dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:35:48.868256 24830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0924321a9a54e29b5eb88c55af805dc [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:48.869128 24830 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0924321a9a54e29b5eb88c55af805dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0924321a9a54e29b5eb88c55af805dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43633 } }
I20260619 20:35:48.869247 24830 leader_election.cc:302] T 00000000000000000000000000000000 P b0924321a9a54e29b5eb88c55af805dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0924321a9a54e29b5eb88c55af805dc; no voters:
I20260619 20:35:48.869439 24830 leader_election.cc:288] T 00000000000000000000000000000000 P b0924321a9a54e29b5eb88c55af805dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:35:48.869485 24835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0924321a9a54e29b5eb88c55af805dc [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:35:48.869650 24835 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0924321a9a54e29b5eb88c55af805dc [term 1 LEADER]: Becoming Leader. State: Replica: b0924321a9a54e29b5eb88c55af805dc, State: Running, Role: LEADER
I20260619 20:35:48.869776 24830 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0924321a9a54e29b5eb88c55af805dc [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:35:48.869796 24835 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0924321a9a54e29b5eb88c55af805dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0924321a9a54e29b5eb88c55af805dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43633 } }
I20260619 20:35:48.870165 24835 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0924321a9a54e29b5eb88c55af805dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0924321a9a54e29b5eb88c55af805dc. Latest consensus state: current_term: 1 leader_uuid: "b0924321a9a54e29b5eb88c55af805dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0924321a9a54e29b5eb88c55af805dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43633 } } }
I20260619 20:35:48.870431 24835 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0924321a9a54e29b5eb88c55af805dc [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:48.870349 24837 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0924321a9a54e29b5eb88c55af805dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0924321a9a54e29b5eb88c55af805dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0924321a9a54e29b5eb88c55af805dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43633 } } }
I20260619 20:35:48.870859 24837 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0924321a9a54e29b5eb88c55af805dc [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:48.871047 24844 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:35:48.871683 24844 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:35:48.873823 24844 catalog_manager.cc:1347] Generated new cluster ID: 1a1fd47fa3434ac2aefe2c0a681b9481
I20260619 20:35:48.873884 24834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:48.867896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32895 (local address 127.0.0.1:43633)
0619 20:35:48.868139 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:48.868148 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:48.868192 (+ 44us) server_negotiation.cc:407] Connection header received
0619 20:35:48.868336 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:48.868345 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:48.868405 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:48.868543 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:48.868894 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:48.870173 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:48.871405 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:48.872100 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:48.872222 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:48.873051 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:48.873055 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:48.873056 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:35:48.873308 (+ 252us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:48.873357 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:35:48.873376 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:48.873476 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:48.873595 (+ 119us) server_negotiation.cc:299] Negotiation successful
0619 20:35:48.873630 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":114,"threads_started":1}
I20260619 20:35:48.873898 24844 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:35:48.875101 4429 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:35:48.884617 24834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:48.880304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45971 (local address 127.0.0.1:43633)
0619 20:35:48.880366 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:48.880368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:48.880567 (+ 199us) server_negotiation.cc:407] Connection header received
0619 20:35:48.880630 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:48.880631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:48.880643 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:48.880722 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:48.880877 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:48.881724 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:48.882899 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:48.883287 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:48.883368 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:48.884317 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:48.884319 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:48.884319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:35:48.884328 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:48.884351 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:35:48.884359 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:48.884425 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:48.884483 (+ 58us) server_negotiation.cc:299] Negotiation successful
0619 20:35:48.884502 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 20:35:48.893837 24844 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:35:48.894737 24844 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:35:48.903853 24844 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0924321a9a54e29b5eb88c55af805dc: Generated new TSK 0
I20260619 20:35:48.904139 24844 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:35:48.910396 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 24762
W20260619 20:35:48.914745 24747 connection.cc:535] client connection to 127.0.0.1:43633 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43633): Connection reset by peer (error 104)
2026-06-19T20:35:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1300 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:35:48 dist-test-slave-4m58 krb5kdc[24861](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:35:48 dist-test-slave-4m58 krb5kdc[24861](info): set up 2 sockets
Jun 19 20:35:48 dist-test-slave-4m58 krb5kdc[24861](info): commencing operation
krb5kdc: starting...
W20260619 20:35:49.884908 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.956s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:35:49 dist-test-slave-4m58 krb5kdc[24861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:35:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:35:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:35:50.016798 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45739
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:35:50.101219 24874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:50.101465 24874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:50.101497 24874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:50.102730 24874 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:35:50.102792 24874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:50.102818 24874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:50.102834 24874 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:35:50.102849 24874 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:35:50.104482 24874 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45739
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:50.104770 24874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:50.105005 24874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:50.110479 24880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:50.110445 24882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:50.110426 24879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:50.110781 24874 server_base.cc:1034] running on GCE node
I20260619 20:35:50.111130 24874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:50.111443 24874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:50.112636 24874 hybrid_clock.cc:648] HybridClock initialized: now 1781901350112613 us; error 58 us; skew 500 ppm
Jun 19 20:35:50 dist-test-slave-4m58 krb5kdc[24861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901350, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:35:50.115437 24874 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:35:50.115991 24874 webserver.cc:460] Webserver started at http://127.0.0.1:40541/ using document root <none> and password file <none>
I20260619 20:35:50.116242 24874 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:50.116295 24874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:50.116431 24874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:50.117542 24874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "c32d979c9e6b4e8a93654b703cc650df"
format_stamp: "Formatted at 2026-06-19 20:35:50 on dist-test-slave-4m58"
server_key: "0213eb52f0287b06cbffad670e231a29"
server_key_iv: "d6c40cb7ad3b91b9678087fbc65e9cdb"
server_key_version: "encryptionkey@0"
I20260619 20:35:50.117888 24874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "c32d979c9e6b4e8a93654b703cc650df"
format_stamp: "Formatted at 2026-06-19 20:35:50 on dist-test-slave-4m58"
server_key: "0213eb52f0287b06cbffad670e231a29"
server_key_iv: "d6c40cb7ad3b91b9678087fbc65e9cdb"
server_key_version: "encryptionkey@0"
I20260619 20:35:50.119424 24874 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:35:50.120393 24889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:50.120616 24874 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:35:50.120713 24874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "c32d979c9e6b4e8a93654b703cc650df"
format_stamp: "Formatted at 2026-06-19 20:35:50 on dist-test-slave-4m58"
server_key: "0213eb52f0287b06cbffad670e231a29"
server_key_iv: "d6c40cb7ad3b91b9678087fbc65e9cdb"
server_key_version: "encryptionkey@0"
I20260619 20:35:50.120785 24874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:50.124305 24874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:50.126657 24874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:50.126811 24874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:50.131201 24874 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45739
I20260619 20:35:50.131343 24941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45739 every 8 connection(s)
I20260619 20:35:50.131646 24874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:35:50.132925 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 24874
I20260619 20:35:50.133091 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:35:50.133126 24942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:50.133342 4429 external_mini_cluster.cc:1428] Setting key 2839c178da02512ce1d5874d24093003
I20260619 20:35:50.136816 24945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:50.134196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55093 (local address 127.0.0.1:45739)
0619 20:35:50.134405 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:50.134415 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:50.134446 (+ 31us) server_negotiation.cc:407] Connection header received
0619 20:35:50.135059 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:50.135066 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:50.135103 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:50.135249 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:50.136332 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:50.136335 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:50.136336 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:35:50.136340 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:50.136389 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:35:50.136400 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:50.136477 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:50.136566 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 20:35:50.136604 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":88,"threads_started":1}
I20260619 20:35:50.138002 4429 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:35:50.138720 24942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c32d979c9e6b4e8a93654b703cc650df: Bootstrap starting.
I20260619 20:35:50.139819 24942 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c32d979c9e6b4e8a93654b703cc650df: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:50.140255 24942 log.cc:824] T 00000000000000000000000000000000 P c32d979c9e6b4e8a93654b703cc650df: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:50.141352 24942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c32d979c9e6b4e8a93654b703cc650df: No bootstrap required, opened a new log
I20260619 20:35:50.141935 24945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:50.140301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35491 (local address 127.0.0.1:45739)
0619 20:35:50.140376 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:50.140379 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:50.140705 (+ 326us) server_negotiation.cc:407] Connection header received
0619 20:35:50.140798 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:50.140800 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:50.140861 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:50.140981 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:50.141671 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:50.141673 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:50.141674 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:35:50.141680 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:50.141704 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:35:50.141709 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:50.141796 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:50.141851 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 20:35:50.141867 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 20:35:50.144248 24942 raft_consensus.cc:348] T 00000000000000000000000000000000 P c32d979c9e6b4e8a93654b703cc650df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c32d979c9e6b4e8a93654b703cc650df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45739 } }
I20260619 20:35:50.144447 24942 raft_consensus.cc:374] T 00000000000000000000000000000000 P c32d979c9e6b4e8a93654b703cc650df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:50.144469 24942 raft_consensus.cc:729] T 00000000000000000000000000000000 P c32d979c9e6b4e8a93654b703cc650df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c32d979c9e6b4e8a93654b703cc650df, State: Initialized, Role: FOLLOWER
I20260619 20:35:50.144595 24942 consensus_queue.cc:260] T 00000000000000000000000000000000 P c32d979c9e6b4e8a93654b703cc650df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c32d979c9e6b4e8a93654b703cc650df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45739 } }
I20260619 20:35:50.144678 24942 raft_consensus.cc:388] T 00000000000000000000000000000000 P c32d979c9e6b4e8a93654b703cc650df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:35:50.144714 24942 raft_consensus.cc:482] T 00000000000000000000000000000000 P c32d979c9e6b4e8a93654b703cc650df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:35:50.144776 24942 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c32d979c9e6b4e8a93654b703cc650df [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:50.145475 24942 raft_consensus.cc:504] T 00000000000000000000000000000000 P c32d979c9e6b4e8a93654b703cc650df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c32d979c9e6b4e8a93654b703cc650df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45739 } }
I20260619 20:35:50.145570 24942 leader_election.cc:302] T 00000000000000000000000000000000 P c32d979c9e6b4e8a93654b703cc650df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c32d979c9e6b4e8a93654b703cc650df; no voters:
I20260619 20:35:50.145860 24952 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c32d979c9e6b4e8a93654b703cc650df [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:35:50.145862 24942 leader_election.cc:288] T 00000000000000000000000000000000 P c32d979c9e6b4e8a93654b703cc650df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:35:50.146044 24952 raft_consensus.cc:686] T 00000000000000000000000000000000 P c32d979c9e6b4e8a93654b703cc650df [term 1 LEADER]: Becoming Leader. State: Replica: c32d979c9e6b4e8a93654b703cc650df, State: Running, Role: LEADER
I20260619 20:35:50.146191 24952 consensus_queue.cc:237] T 00000000000000000000000000000000 P c32d979c9e6b4e8a93654b703cc650df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c32d979c9e6b4e8a93654b703cc650df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45739 } }
I20260619 20:35:50.146579 24942 sys_catalog.cc:564] T 00000000000000000000000000000000 P c32d979c9e6b4e8a93654b703cc650df [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:35:50.146615 24954 sys_catalog.cc:455] T 00000000000000000000000000000000 P c32d979c9e6b4e8a93654b703cc650df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c32d979c9e6b4e8a93654b703cc650df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c32d979c9e6b4e8a93654b703cc650df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45739 } } }
I20260619 20:35:50.146972 24954 sys_catalog.cc:458] T 00000000000000000000000000000000 P c32d979c9e6b4e8a93654b703cc650df [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:50.147228 24956 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:35:50.147779 24956 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:35:50.146626 24953 sys_catalog.cc:455] T 00000000000000000000000000000000 P c32d979c9e6b4e8a93654b703cc650df [sys.catalog]: SysCatalogTable state changed. Reason: New leader c32d979c9e6b4e8a93654b703cc650df. Latest consensus state: current_term: 1 leader_uuid: "c32d979c9e6b4e8a93654b703cc650df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c32d979c9e6b4e8a93654b703cc650df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45739 } } }
I20260619 20:35:50.147930 24953 sys_catalog.cc:458] T 00000000000000000000000000000000 P c32d979c9e6b4e8a93654b703cc650df [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:50.149609 24956 catalog_manager.cc:1347] Generated new cluster ID: db5666fed7d042c69630b85b2845e4cf
I20260619 20:35:50.149672 24956 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:35:50.180447 24956 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:35:50.180742 24956 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:35:50.199976 24956 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c32d979c9e6b4e8a93654b703cc650df: Generated new TSK 0
I20260619 20:35:50.200280 24956 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:35:50.203006 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 24874
2026-06-19T20:35:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1293 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:35:50 dist-test-slave-4m58 krb5kdc[24973](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:35:50 dist-test-slave-4m58 krb5kdc[24973](info): set up 2 sockets
Jun 19 20:35:50 dist-test-slave-4m58 krb5kdc[24973](info): commencing operation
krb5kdc: starting...
W20260619 20:35:51.169293 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.947s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:35:51 dist-test-slave-4m58 krb5kdc[24973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:35:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:35:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:35:51.303818 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43419
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:35:51.401841 24986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:51.402166 24986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:51.402208 24986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:51.403769 24986 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:35:51.403898 24986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:51.403923 24986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:51.403939 24986 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:35:51.403956 24986 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:35:51.406105 24986 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:51.406515 24986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:51.406795 24986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:51.412915 24992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:51.412919 24991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:51.413015 24986 server_base.cc:1034] running on GCE node
W20260619 20:35:51.412919 24994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:51.413512 24986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:51.413905 24986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:51.415086 24986 hybrid_clock.cc:648] HybridClock initialized: now 1781901351415063 us; error 43 us; skew 500 ppm
Jun 19 20:35:51 dist-test-slave-4m58 krb5kdc[24973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901351, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:35:51.418232 24986 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:35:51.418828 24986 webserver.cc:460] Webserver started at http://127.0.0.1:40891/ using document root <none> and password file <none>
I20260619 20:35:51.419098 24986 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:51.419157 24986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:51.419283 24986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:51.420681 24986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "227aeae5ebb9497393622c50bd802801"
format_stamp: "Formatted at 2026-06-19 20:35:51 on dist-test-slave-4m58"
server_key: "fd1ee4d7de941c416e457b934c0ca4ee"
server_key_iv: "d73627c7148b3c97708071afdf257e9d"
server_key_version: "encryptionkey@0"
I20260619 20:35:51.421077 24986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "227aeae5ebb9497393622c50bd802801"
format_stamp: "Formatted at 2026-06-19 20:35:51 on dist-test-slave-4m58"
server_key: "fd1ee4d7de941c416e457b934c0ca4ee"
server_key_iv: "d73627c7148b3c97708071afdf257e9d"
server_key_version: "encryptionkey@0"
I20260619 20:35:51.422703 24986 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 20:35:51.423733 25001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:51.423978 24986 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:51.424088 24986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "227aeae5ebb9497393622c50bd802801"
format_stamp: "Formatted at 2026-06-19 20:35:51 on dist-test-slave-4m58"
server_key: "fd1ee4d7de941c416e457b934c0ca4ee"
server_key_iv: "d73627c7148b3c97708071afdf257e9d"
server_key_version: "encryptionkey@0"
I20260619 20:35:51.424189 24986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:51.440124 24986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:51.442691 24986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:51.442829 24986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:51.447312 24986 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43419
I20260619 20:35:51.447365 25053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43419 every 8 connection(s)
I20260619 20:35:51.447767 24986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:35:51.448925 25054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:51.450515 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 24986
I20260619 20:35:51.450639 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:35:51.450802 4429 external_mini_cluster.cc:1428] Setting key d734cefdf4be366b446f51b966268ec4
I20260619 20:35:51.451984 25054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 227aeae5ebb9497393622c50bd802801: Bootstrap starting.
I20260619 20:35:51.452822 25054 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 227aeae5ebb9497393622c50bd802801: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:51.453141 25054 log.cc:824] T 00000000000000000000000000000000 P 227aeae5ebb9497393622c50bd802801: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:51.453888 25054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 227aeae5ebb9497393622c50bd802801: No bootstrap required, opened a new log
I20260619 20:35:51.456270 25054 raft_consensus.cc:348] T 00000000000000000000000000000000 P 227aeae5ebb9497393622c50bd802801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "227aeae5ebb9497393622c50bd802801" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43419 } }
I20260619 20:35:51.456470 25054 raft_consensus.cc:374] T 00000000000000000000000000000000 P 227aeae5ebb9497393622c50bd802801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:51.456506 25054 raft_consensus.cc:729] T 00000000000000000000000000000000 P 227aeae5ebb9497393622c50bd802801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227aeae5ebb9497393622c50bd802801, State: Initialized, Role: FOLLOWER
I20260619 20:35:51.456655 25054 consensus_queue.cc:260] T 00000000000000000000000000000000 P 227aeae5ebb9497393622c50bd802801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "227aeae5ebb9497393622c50bd802801" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43419 } }
I20260619 20:35:51.456744 25054 raft_consensus.cc:388] T 00000000000000000000000000000000 P 227aeae5ebb9497393622c50bd802801 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:35:51.456791 25054 raft_consensus.cc:482] T 00000000000000000000000000000000 P 227aeae5ebb9497393622c50bd802801 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:35:51.456848 25054 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 227aeae5ebb9497393622c50bd802801 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:51.457592 25054 raft_consensus.cc:504] T 00000000000000000000000000000000 P 227aeae5ebb9497393622c50bd802801 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "227aeae5ebb9497393622c50bd802801" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43419 } }
I20260619 20:35:51.457712 25054 leader_election.cc:302] T 00000000000000000000000000000000 P 227aeae5ebb9497393622c50bd802801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 227aeae5ebb9497393622c50bd802801; no voters:
I20260619 20:35:51.457918 25054 leader_election.cc:288] T 00000000000000000000000000000000 P 227aeae5ebb9497393622c50bd802801 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:35:51.457962 25059 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 227aeae5ebb9497393622c50bd802801 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:35:51.458091 25059 raft_consensus.cc:686] T 00000000000000000000000000000000 P 227aeae5ebb9497393622c50bd802801 [term 1 LEADER]: Becoming Leader. State: Replica: 227aeae5ebb9497393622c50bd802801, State: Running, Role: LEADER
I20260619 20:35:51.458241 25059 consensus_queue.cc:237] T 00000000000000000000000000000000 P 227aeae5ebb9497393622c50bd802801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "227aeae5ebb9497393622c50bd802801" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43419 } }
I20260619 20:35:51.458284 25054 sys_catalog.cc:564] T 00000000000000000000000000000000 P 227aeae5ebb9497393622c50bd802801 [sys.catalog]: configured and running, proceeding with master startup.
Jun 19 20:35:51 dist-test-slave-4m58 krb5kdc[24973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 20:35:51.458737 25059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 227aeae5ebb9497393622c50bd802801 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 227aeae5ebb9497393622c50bd802801. Latest consensus state: current_term: 1 leader_uuid: "227aeae5ebb9497393622c50bd802801" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "227aeae5ebb9497393622c50bd802801" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43419 } } }
I20260619 20:35:51.458833 25059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 227aeae5ebb9497393622c50bd802801 [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:51.459051 25060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 227aeae5ebb9497393622c50bd802801 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "227aeae5ebb9497393622c50bd802801" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "227aeae5ebb9497393622c50bd802801" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43419 } } }
I20260619 20:35:51.459153 25060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 227aeae5ebb9497393622c50bd802801 [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:51.459991 25068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:35:51.461024 25068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:35:51.463078 25068 catalog_manager.cc:1347] Generated new cluster ID: 78e15003c7db426283f154fdbf8e8bb7
I20260619 20:35:51.463145 25068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:35:51.464399 25057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:51.451417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41269 (local address 127.0.0.1:43419)
0619 20:35:51.451629 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:51.451641 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:51.451677 (+ 36us) server_negotiation.cc:407] Connection header received
0619 20:35:51.452313 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:51.452327 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:51.452413 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:51.452601 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:51.453313 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:51.454415 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:51.455426 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:51.455992 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:51.456110 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:51.459051 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:51.459058 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:51.459058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:35:51.459076 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:51.461613 (+ 2537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:51.462209 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:51.462214 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:51.462215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:51.462341 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:51.462593 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:51.462594 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:51.462595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:51.463016 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:51.463795 (+ 779us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:51.463938 (+ 143us) server_negotiation.cc:299] Negotiation successful
0619 20:35:51.464138 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":94,"threads_started":1}
I20260619 20:35:51.465615 4429 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:35:51.468225 25068 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:35:51.468792 25068 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:35:51.474077 25068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 227aeae5ebb9497393622c50bd802801: Generated new TSK 0
I20260619 20:35:51.474395 25068 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:35:51.476724 25057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:51.467510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49699 (local address 127.0.0.1:43419)
0619 20:35:51.467556 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:51.467558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:51.467597 (+ 39us) server_negotiation.cc:407] Connection header received
0619 20:35:51.467674 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:51.467675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:51.467687 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:51.467745 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:51.468618 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:51.469431 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:51.470642 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:51.471030 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:51.471095 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:51.473343 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:51.473345 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:51.473346 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:35:51.473357 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:51.475302 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:51.475790 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:51.475791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:51.475792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:51.475901 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:51.476107 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:51.476109 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:51.476109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:51.476278 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:51.476382 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:51.476472 (+ 90us) server_negotiation.cc:299] Negotiation successful
0619 20:35:51.476607 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:35:51.479120 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 24986
2026-06-19T20:35:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1275 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:35:51 dist-test-slave-4m58 krb5kdc[25085](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:35:51 dist-test-slave-4m58 krb5kdc[25085](info): set up 2 sockets
Jun 19 20:35:51 dist-test-slave-4m58 krb5kdc[25085](info): commencing operation
krb5kdc: starting...
W20260619 20:35:52.453126 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.955s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:35:52 dist-test-slave-4m58 krb5kdc[25085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:35:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:35:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:35:52.591614 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46713
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:35:52.696628 25098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:52.696920 25098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:52.696951 25098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:52.698770 25098 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:35:52.698865 25098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:52.698881 25098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:52.698894 25098 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:35:52.698906 25098 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:35:52.701263 25098 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:52.701948 25098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:52.702307 25098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:52.708210 25104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:52.708191 25103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:52.708197 25106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:52.708235 25098 server_base.cc:1034] running on GCE node
I20260619 20:35:52.708801 25098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:52.709174 25098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:52.710371 25098 hybrid_clock.cc:648] HybridClock initialized: now 1781901352710333 us; error 54 us; skew 500 ppm
Jun 19 20:35:52 dist-test-slave-4m58 krb5kdc[25085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901352, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:35:52.713866 25098 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:35:52.714689 25098 webserver.cc:460] Webserver started at http://127.0.0.1:43297/ using document root <none> and password file <none>
I20260619 20:35:52.715034 25098 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:52.715099 25098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:52.715260 25098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:52.716773 25098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "1f6a97783af54b1798edc208cf64d82f"
format_stamp: "Formatted at 2026-06-19 20:35:52 on dist-test-slave-4m58"
server_key: "be33d6c79a9b982f6d73fddd80f78721"
server_key_iv: "a941a2576ea8f831fd6e399711c585f4"
server_key_version: "encryptionkey@0"
I20260619 20:35:52.717227 25098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "1f6a97783af54b1798edc208cf64d82f"
format_stamp: "Formatted at 2026-06-19 20:35:52 on dist-test-slave-4m58"
server_key: "be33d6c79a9b982f6d73fddd80f78721"
server_key_iv: "a941a2576ea8f831fd6e399711c585f4"
server_key_version: "encryptionkey@0"
I20260619 20:35:52.719585 25098 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260619 20:35:52.720875 25113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:52.721243 25098 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:52.721392 25098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "1f6a97783af54b1798edc208cf64d82f"
format_stamp: "Formatted at 2026-06-19 20:35:52 on dist-test-slave-4m58"
server_key: "be33d6c79a9b982f6d73fddd80f78721"
server_key_iv: "a941a2576ea8f831fd6e399711c585f4"
server_key_version: "encryptionkey@0"
I20260619 20:35:52.721503 25098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:52.747351 25098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:52.749902 25098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:52.750062 25098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:52.754944 25098 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46713
I20260619 20:35:52.755080 25165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46713 every 8 connection(s)
I20260619 20:35:52.755429 25098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:35:52.756625 25166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:52.758512 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 25098
I20260619 20:35:52.758634 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:35:52.758808 4429 external_mini_cluster.cc:1428] Setting key 9419fcedb0b1b2054759d7f7aaddad0b
I20260619 20:35:52.759150 25166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f6a97783af54b1798edc208cf64d82f: Bootstrap starting.
I20260619 20:35:52.759933 25166 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f6a97783af54b1798edc208cf64d82f: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:52.760278 25166 log.cc:824] T 00000000000000000000000000000000 P 1f6a97783af54b1798edc208cf64d82f: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:52.761242 25166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f6a97783af54b1798edc208cf64d82f: No bootstrap required, opened a new log
I20260619 20:35:52.763512 25166 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f6a97783af54b1798edc208cf64d82f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f6a97783af54b1798edc208cf64d82f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46713 } }
I20260619 20:35:52.763703 25166 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f6a97783af54b1798edc208cf64d82f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:52.763738 25166 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f6a97783af54b1798edc208cf64d82f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f6a97783af54b1798edc208cf64d82f, State: Initialized, Role: FOLLOWER
I20260619 20:35:52.763994 25166 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f6a97783af54b1798edc208cf64d82f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f6a97783af54b1798edc208cf64d82f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46713 } }
I20260619 20:35:52.764093 25166 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f6a97783af54b1798edc208cf64d82f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:35:52.764163 25166 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f6a97783af54b1798edc208cf64d82f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:35:52.764235 25166 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f6a97783af54b1798edc208cf64d82f [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:52.764981 25166 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f6a97783af54b1798edc208cf64d82f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f6a97783af54b1798edc208cf64d82f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46713 } }
I20260619 20:35:52.765079 25166 leader_election.cc:302] T 00000000000000000000000000000000 P 1f6a97783af54b1798edc208cf64d82f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f6a97783af54b1798edc208cf64d82f; no voters:
I20260619 20:35:52.765276 25166 leader_election.cc:288] T 00000000000000000000000000000000 P 1f6a97783af54b1798edc208cf64d82f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:35:52.765460 25172 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f6a97783af54b1798edc208cf64d82f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:35:52.765561 25166 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f6a97783af54b1798edc208cf64d82f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:35:52.765739 25172 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f6a97783af54b1798edc208cf64d82f [term 1 LEADER]: Becoming Leader. State: Replica: 1f6a97783af54b1798edc208cf64d82f, State: Running, Role: LEADER
I20260619 20:35:52.765784 25169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:52.759422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35193 (local address 127.0.0.1:46713)
0619 20:35:52.759662 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:52.759672 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:52.759721 (+ 49us) server_negotiation.cc:407] Connection header received
0619 20:35:52.760368 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:52.760378 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:52.760444 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:52.760614 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:52.760927 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:52.762343 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:52.763206 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:52.763908 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:52.764446 (+ 538us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:52.764927 (+ 481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:52.764932 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:52.764933 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:35:52.765164 (+ 231us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:52.765213 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:35:52.765229 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:52.765281 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:52.765404 (+ 123us) server_negotiation.cc:299] Negotiation successful
0619 20:35:52.765499 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":97,"threads_started":1}
I20260619 20:35:52.765898 25172 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f6a97783af54b1798edc208cf64d82f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f6a97783af54b1798edc208cf64d82f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46713 } }
I20260619 20:35:52.766479 25173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f6a97783af54b1798edc208cf64d82f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f6a97783af54b1798edc208cf64d82f. Latest consensus state: current_term: 1 leader_uuid: "1f6a97783af54b1798edc208cf64d82f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f6a97783af54b1798edc208cf64d82f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46713 } } }
I20260619 20:35:52.766589 25173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f6a97783af54b1798edc208cf64d82f [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:52.766840 4429 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:35:52.766889 25180 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:35:52.767109 25171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f6a97783af54b1798edc208cf64d82f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f6a97783af54b1798edc208cf64d82f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f6a97783af54b1798edc208cf64d82f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46713 } } }
I20260619 20:35:52.767225 25171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f6a97783af54b1798edc208cf64d82f [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:52.767457 25180 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:35:52.769590 25180 catalog_manager.cc:1347] Generated new cluster ID: ae7fb95fb22b4bf0a8a5eb8a6943043b
I20260619 20:35:52.769649 25180 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:35:52.772221 25169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:52.768587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48131 (local address 127.0.0.1:46713)
0619 20:35:52.768916 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:52.768918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:52.768925 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:35:52.768943 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:52.768944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:52.768952 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:52.768997 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:52.769136 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:52.769724 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:52.770758 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:52.771057 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:52.771093 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:52.771969 (+ 876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:52.771970 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:52.771970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:35:52.771975 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:52.771993 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:35:52.771998 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:52.772024 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:52.772113 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 20:35:52.772141 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312}
I20260619 20:35:52.787287 25180 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:35:52.788232 25180 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:35:52.795413 25180 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f6a97783af54b1798edc208cf64d82f: Generated new TSK 0
I20260619 20:35:52.795652 25180 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:35:52.798848 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 25098
2026-06-19T20:35:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1319 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:35:52 dist-test-slave-4m58 krb5kdc[25197](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:35:52 dist-test-slave-4m58 krb5kdc[25197](info): set up 2 sockets
Jun 19 20:35:52 dist-test-slave-4m58 krb5kdc[25197](info): commencing operation
krb5kdc: starting...
W20260619 20:35:53.793128 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.975s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:35:53 dist-test-slave-4m58 krb5kdc[25197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:35:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:35:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:35:53.932179 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46271
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46271
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:35:54.023492 25210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:54.023774 25210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:54.023800 25210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:54.025188 25210 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:35:54.025267 25210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:54.025282 25210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:54.025295 25210 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:35:54.025305 25210 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:35:54.027326 25210 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46271
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:54.027621 25210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:54.027916 25210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:54.034500 25216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:54.034741 25210 server_base.cc:1034] running on GCE node
W20260619 20:35:54.034586 25215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:54.034586 25218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:54.035247 25210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:54.035676 25210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:54.036891 25210 hybrid_clock.cc:648] HybridClock initialized: now 1781901354036860 us; error 53 us; skew 500 ppm
Jun 19 20:35:54 dist-test-slave-4m58 krb5kdc[25197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901354, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:35:54.047194 25210 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:35:54.047950 25210 webserver.cc:460] Webserver started at http://127.0.0.1:35699/ using document root <none> and password file <none>
I20260619 20:35:54.048254 25210 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:54.048316 25210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:54.048478 25210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:54.049911 25210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "622a4f2847a44accbe1cec6199d96c91"
format_stamp: "Formatted at 2026-06-19 20:35:54 on dist-test-slave-4m58"
server_key: "fcd4a316da17ce75f61a924fc792256e"
server_key_iv: "5fd4b4ecd1a3ceb6eda14b371eeda943"
server_key_version: "encryptionkey@0"
I20260619 20:35:54.050375 25210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "622a4f2847a44accbe1cec6199d96c91"
format_stamp: "Formatted at 2026-06-19 20:35:54 on dist-test-slave-4m58"
server_key: "fcd4a316da17ce75f61a924fc792256e"
server_key_iv: "5fd4b4ecd1a3ceb6eda14b371eeda943"
server_key_version: "encryptionkey@0"
I20260619 20:35:54.052475 25210 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260619 20:35:54.053964 25225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:54.054397 25210 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:54.054610 25210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "622a4f2847a44accbe1cec6199d96c91"
format_stamp: "Formatted at 2026-06-19 20:35:54 on dist-test-slave-4m58"
server_key: "fcd4a316da17ce75f61a924fc792256e"
server_key_iv: "5fd4b4ecd1a3ceb6eda14b371eeda943"
server_key_version: "encryptionkey@0"
I20260619 20:35:54.054735 25210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:54.059448 25210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:54.062813 25210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:54.063069 25210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:54.069218 25210 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46271
I20260619 20:35:54.069386 25277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46271 every 8 connection(s)
I20260619 20:35:54.069754 25210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:35:54.071614 25278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:54.075428 25278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 622a4f2847a44accbe1cec6199d96c91: Bootstrap starting.
I20260619 20:35:54.076263 25278 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 622a4f2847a44accbe1cec6199d96c91: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:54.076545 25278 log.cc:824] T 00000000000000000000000000000000 P 622a4f2847a44accbe1cec6199d96c91: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:54.077574 25278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 622a4f2847a44accbe1cec6199d96c91: No bootstrap required, opened a new log
I20260619 20:35:54.078802 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 25210
I20260619 20:35:54.078941 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:35:54.079139 4429 external_mini_cluster.cc:1428] Setting key d6fe893cf03de45fdc30b865edb80f44
I20260619 20:35:54.079982 25278 raft_consensus.cc:348] T 00000000000000000000000000000000 P 622a4f2847a44accbe1cec6199d96c91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622a4f2847a44accbe1cec6199d96c91" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46271 } }
I20260619 20:35:54.080181 25278 raft_consensus.cc:374] T 00000000000000000000000000000000 P 622a4f2847a44accbe1cec6199d96c91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:54.080436 25278 raft_consensus.cc:729] T 00000000000000000000000000000000 P 622a4f2847a44accbe1cec6199d96c91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 622a4f2847a44accbe1cec6199d96c91, State: Initialized, Role: FOLLOWER
I20260619 20:35:54.080624 25278 consensus_queue.cc:260] T 00000000000000000000000000000000 P 622a4f2847a44accbe1cec6199d96c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622a4f2847a44accbe1cec6199d96c91" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46271 } }
I20260619 20:35:54.080781 25278 raft_consensus.cc:388] T 00000000000000000000000000000000 P 622a4f2847a44accbe1cec6199d96c91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:35:54.080821 25278 raft_consensus.cc:482] T 00000000000000000000000000000000 P 622a4f2847a44accbe1cec6199d96c91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:35:54.080883 25278 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 622a4f2847a44accbe1cec6199d96c91 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:54.082158 25278 raft_consensus.cc:504] T 00000000000000000000000000000000 P 622a4f2847a44accbe1cec6199d96c91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622a4f2847a44accbe1cec6199d96c91" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46271 } }
I20260619 20:35:54.082268 25282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:54.080042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58639 (local address 127.0.0.1:46271)
0619 20:35:54.080329 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:54.080341 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:54.080379 (+ 38us) server_negotiation.cc:407] Connection header received
0619 20:35:54.080540 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:54.080551 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:54.080595 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:54.080747 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:54.081658 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:54.081663 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:54.081663 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:35:54.081668 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:54.081748 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:35:54.081764 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:54.081835 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:54.081937 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 20:35:54.082002 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":153,"threads_started":1}
I20260619 20:35:54.082274 25278 leader_election.cc:302] T 00000000000000000000000000000000 P 622a4f2847a44accbe1cec6199d96c91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 622a4f2847a44accbe1cec6199d96c91; no voters:
I20260619 20:35:54.082582 25278 leader_election.cc:288] T 00000000000000000000000000000000 P 622a4f2847a44accbe1cec6199d96c91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:35:54.082831 25283 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 622a4f2847a44accbe1cec6199d96c91 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:35:54.083209 25278 sys_catalog.cc:564] T 00000000000000000000000000000000 P 622a4f2847a44accbe1cec6199d96c91 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:35:54.083386 25283 raft_consensus.cc:686] T 00000000000000000000000000000000 P 622a4f2847a44accbe1cec6199d96c91 [term 1 LEADER]: Becoming Leader. State: Replica: 622a4f2847a44accbe1cec6199d96c91, State: Running, Role: LEADER
I20260619 20:35:54.083542 25283 consensus_queue.cc:237] T 00000000000000000000000000000000 P 622a4f2847a44accbe1cec6199d96c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622a4f2847a44accbe1cec6199d96c91" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46271 } }
I20260619 20:35:54.083901 4429 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260619 20:35:54.085680 25297 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 622a4f2847a44accbe1cec6199d96c91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:35:54.085776 25297 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:35:54.085950 25284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 622a4f2847a44accbe1cec6199d96c91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "622a4f2847a44accbe1cec6199d96c91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622a4f2847a44accbe1cec6199d96c91" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46271 } } }
I20260619 20:35:54.086081 25284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 622a4f2847a44accbe1cec6199d96c91 [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:54.086490 25285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 622a4f2847a44accbe1cec6199d96c91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 622a4f2847a44accbe1cec6199d96c91. Latest consensus state: current_term: 1 leader_uuid: "622a4f2847a44accbe1cec6199d96c91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622a4f2847a44accbe1cec6199d96c91" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46271 } } }
I20260619 20:35:54.086561 25285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 622a4f2847a44accbe1cec6199d96c91 [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:54.086692 25305 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:35:54.086921 25305 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:35:54.087549 25282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:54.085792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58767 (local address 127.0.0.1:46271)
0619 20:35:54.085857 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:54.085860 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:54.086203 (+ 343us) server_negotiation.cc:407] Connection header received
0619 20:35:54.086279 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:54.086281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:54.086289 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:54.086365 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:54.087264 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:54.087265 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:54.087266 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:35:54.087271 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:54.087293 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:35:54.087300 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:54.087372 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:54.087454 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 20:35:54.087471 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 20:35:54.089258 25305 catalog_manager.cc:1347] Generated new cluster ID: 87333905180f4f06ad36825474a89ca4
I20260619 20:35:54.089375 25305 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:35:54.097523 25305 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:35:54.097820 25305 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:35:54.103611 25305 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 622a4f2847a44accbe1cec6199d96c91: Generated new TSK 0
I20260619 20:35:54.103962 25305 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:35:54.108100 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 25210
2026-06-19T20:35:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1308 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:35:54 dist-test-slave-4m58 krb5kdc[25309](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:35:54 dist-test-slave-4m58 krb5kdc[25309](info): set up 2 sockets
Jun 19 20:35:54 dist-test-slave-4m58 krb5kdc[25309](info): commencing operation
krb5kdc: starting...
W20260619 20:35:55.093797 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.968s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:35:55 dist-test-slave-4m58 krb5kdc[25309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:35:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:35:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:35:55.248750 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41679
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41679
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:35:55.356175 25322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:55.356487 25322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:55.356529 25322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:55.358083 25322 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:35:55.358157 25322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:55.358177 25322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:55.358194 25322 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:35:55.358210 25322 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:35:55.360780 25322 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41679
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:55.361356 25322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:55.361698 25322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:55.368175 25330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:55.368175 25327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:55.368180 25328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:55.368391 25322 server_base.cc:1034] running on GCE node
I20260619 20:35:55.368884 25322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:55.369297 25322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:55.370545 25322 hybrid_clock.cc:648] HybridClock initialized: now 1781901355370551 us; error 75 us; skew 500 ppm
Jun 19 20:35:55 dist-test-slave-4m58 krb5kdc[25309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901355, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:35:55.374099 25322 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:35:55.374754 25322 webserver.cc:460] Webserver started at http://127.0.0.1:37019/ using document root <none> and password file <none>
I20260619 20:35:55.375062 25322 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:55.375130 25322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:55.375284 25322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:55.376762 25322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "bacb871df9d34919af5b9dafa57560af"
format_stamp: "Formatted at 2026-06-19 20:35:55 on dist-test-slave-4m58"
server_key: "175edd242770316acc1bfc42246566fe"
server_key_iv: "31e5e2e470889c6ffe7f82ed2137a718"
server_key_version: "encryptionkey@0"
I20260619 20:35:55.377180 25322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "bacb871df9d34919af5b9dafa57560af"
format_stamp: "Formatted at 2026-06-19 20:35:55 on dist-test-slave-4m58"
server_key: "175edd242770316acc1bfc42246566fe"
server_key_iv: "31e5e2e470889c6ffe7f82ed2137a718"
server_key_version: "encryptionkey@0"
I20260619 20:35:55.379050 25322 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260619 20:35:55.379949 25337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:55.380187 25322 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:55.380264 25322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "bacb871df9d34919af5b9dafa57560af"
format_stamp: "Formatted at 2026-06-19 20:35:55 on dist-test-slave-4m58"
server_key: "175edd242770316acc1bfc42246566fe"
server_key_iv: "31e5e2e470889c6ffe7f82ed2137a718"
server_key_version: "encryptionkey@0"
I20260619 20:35:55.380347 25322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:55.404304 25322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:55.407756 25322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:55.407995 25322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:55.413417 25322 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41679
I20260619 20:35:55.413820 25389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41679 every 8 connection(s)
I20260619 20:35:55.413967 25322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:35:55.414887 25390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:55.416401 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 25322
I20260619 20:35:55.416555 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:35:55.416744 4429 external_mini_cluster.cc:1428] Setting key 3d74f70e0d5a1b40e631d6680e4f4cd4
I20260619 20:35:55.418753 25390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bacb871df9d34919af5b9dafa57560af: Bootstrap starting.
I20260619 20:35:55.419585 25390 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bacb871df9d34919af5b9dafa57560af: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:55.420022 25390 log.cc:824] T 00000000000000000000000000000000 P bacb871df9d34919af5b9dafa57560af: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:55.421394 25390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bacb871df9d34919af5b9dafa57560af: No bootstrap required, opened a new log
I20260619 20:35:55.424172 25390 raft_consensus.cc:348] T 00000000000000000000000000000000 P bacb871df9d34919af5b9dafa57560af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bacb871df9d34919af5b9dafa57560af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41679 } }
I20260619 20:35:55.424444 25390 raft_consensus.cc:374] T 00000000000000000000000000000000 P bacb871df9d34919af5b9dafa57560af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:55.424480 25390 raft_consensus.cc:729] T 00000000000000000000000000000000 P bacb871df9d34919af5b9dafa57560af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bacb871df9d34919af5b9dafa57560af, State: Initialized, Role: FOLLOWER
I20260619 20:35:55.424634 25390 consensus_queue.cc:260] T 00000000000000000000000000000000 P bacb871df9d34919af5b9dafa57560af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bacb871df9d34919af5b9dafa57560af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41679 } }
I20260619 20:35:55.424726 25390 raft_consensus.cc:388] T 00000000000000000000000000000000 P bacb871df9d34919af5b9dafa57560af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:35:55.424757 25390 raft_consensus.cc:482] T 00000000000000000000000000000000 P bacb871df9d34919af5b9dafa57560af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:35:55.424814 25390 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bacb871df9d34919af5b9dafa57560af [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:55.425627 25390 raft_consensus.cc:504] T 00000000000000000000000000000000 P bacb871df9d34919af5b9dafa57560af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bacb871df9d34919af5b9dafa57560af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41679 } }
I20260619 20:35:55.425776 25390 leader_election.cc:302] T 00000000000000000000000000000000 P bacb871df9d34919af5b9dafa57560af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bacb871df9d34919af5b9dafa57560af; no voters:
I20260619 20:35:55.426028 25390 leader_election.cc:288] T 00000000000000000000000000000000 P bacb871df9d34919af5b9dafa57560af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:35:55.426086 25395 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bacb871df9d34919af5b9dafa57560af [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:35:55.426286 25395 raft_consensus.cc:686] T 00000000000000000000000000000000 P bacb871df9d34919af5b9dafa57560af [term 1 LEADER]: Becoming Leader. State: Replica: bacb871df9d34919af5b9dafa57560af, State: Running, Role: LEADER
Jun 19 20:35:55 dist-test-slave-4m58 krb5kdc[25309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 20:35:55.426489 25390 sys_catalog.cc:564] T 00000000000000000000000000000000 P bacb871df9d34919af5b9dafa57560af [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:35:55.426595 25395 consensus_queue.cc:237] T 00000000000000000000000000000000 P bacb871df9d34919af5b9dafa57560af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bacb871df9d34919af5b9dafa57560af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41679 } }
I20260619 20:35:55.427057 25395 sys_catalog.cc:455] T 00000000000000000000000000000000 P bacb871df9d34919af5b9dafa57560af [sys.catalog]: SysCatalogTable state changed. Reason: New leader bacb871df9d34919af5b9dafa57560af. Latest consensus state: current_term: 1 leader_uuid: "bacb871df9d34919af5b9dafa57560af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bacb871df9d34919af5b9dafa57560af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41679 } } }
I20260619 20:35:55.427145 25395 sys_catalog.cc:458] T 00000000000000000000000000000000 P bacb871df9d34919af5b9dafa57560af [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:55.427371 25401 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:35:55.427102 25396 sys_catalog.cc:455] T 00000000000000000000000000000000 P bacb871df9d34919af5b9dafa57560af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bacb871df9d34919af5b9dafa57560af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bacb871df9d34919af5b9dafa57560af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41679 } } }
I20260619 20:35:55.428062 25401 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:35:55.428719 25396 sys_catalog.cc:458] T 00000000000000000000000000000000 P bacb871df9d34919af5b9dafa57560af [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:55.431068 25401 catalog_manager.cc:1347] Generated new cluster ID: 0fc0c3089416400b895d6273c94e0279
I20260619 20:35:55.431145 25401 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:35:55.433095 25393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:55.417433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:33683 (local address 127.0.0.1:41679)
0619 20:35:55.417645 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:55.417656 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:55.417688 (+ 32us) server_negotiation.cc:407] Connection header received
0619 20:35:55.418474 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:55.418486 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:55.418799 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:55.419032 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:55.419904 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:55.421259 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:55.422592 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:55.423333 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:55.423528 (+ 195us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:55.427249 (+ 3721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:55.427256 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:55.427256 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:35:55.427280 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:55.430401 (+ 3121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:55.431249 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:55.431262 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:55.431262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:55.431410 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:55.431752 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:55.431752 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:55.431753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:55.432196 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:55.432422 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:55.432635 (+ 213us) server_negotiation.cc:299] Negotiation successful
0619 20:35:55.432783 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":91,"threads_started":1}
I20260619 20:35:55.434674 4429 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:35:55.445413 25401 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:35:55.446265 25401 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:35:55.451295 25401 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bacb871df9d34919af5b9dafa57560af: Generated new TSK 0
I20260619 20:35:55.451609 25401 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:35:55.452096 25393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:55.440383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:50885 (local address 127.0.0.1:41679)
0619 20:35:55.440468 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:55.440471 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:55.440827 (+ 356us) server_negotiation.cc:407] Connection header received
0619 20:35:55.440876 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:55.440877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:55.440892 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:55.440962 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:55.442843 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:55.443705 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:55.445691 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:55.446040 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:55.446134 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:55.448209 (+ 2075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:55.448211 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:55.448212 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:35:55.448223 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:55.450025 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:55.450779 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:55.450781 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:55.450781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:55.450876 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:55.451196 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:55.451197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:55.451197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:55.451365 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:55.451485 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:55.451856 (+ 371us) server_negotiation.cc:299] Negotiation successful
0619 20:35:55.451958 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 20:35:55.455379 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 25322
W20260619 20:35:55.461302 25307 connection.cc:535] client connection to 127.0.0.1:41679 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41679): Connection reset by peer (error 104)
2026-06-19T20:35:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1349 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:35:55 dist-test-slave-4m58 krb5kdc[25421](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:35:55 dist-test-slave-4m58 krb5kdc[25421](info): set up 2 sockets
Jun 19 20:35:55 dist-test-slave-4m58 krb5kdc[25421](info): commencing operation
krb5kdc: starting...
W20260619 20:35:56.453059 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.977s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:35:56 dist-test-slave-4m58 krb5kdc[25421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:35:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:35:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:35:56.572731 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35759
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35759
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:35:56.656251 25434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:56.656486 25434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:56.656518 25434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:56.657720 25434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:35:56.657768 25434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:56.657788 25434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:56.657802 25434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:35:56.657819 25434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:35:56.659415 25434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35759
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:56.659761 25434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:56.660071 25434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:56.665200 25440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:56.665427 25434 server_base.cc:1034] running on GCE node
W20260619 20:35:56.665205 25439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:56.665200 25442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:56.665792 25434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:56.666213 25434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:56.667407 25434 hybrid_clock.cc:648] HybridClock initialized: now 1781901356667376 us; error 48 us; skew 500 ppm
Jun 19 20:35:56 dist-test-slave-4m58 krb5kdc[25421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901356, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:35:56.670601 25434 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:35:56.671100 25434 webserver.cc:460] Webserver started at http://127.0.0.1:41557/ using document root <none> and password file <none>
I20260619 20:35:56.671329 25434 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:56.671380 25434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:56.671504 25434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:56.672645 25434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "1482af400250477a8996f2fe0beeca41"
format_stamp: "Formatted at 2026-06-19 20:35:56 on dist-test-slave-4m58"
server_key: "702b9cdaeda52e4f729ec13d622fdc92"
server_key_iv: "9d6200a9c96ccd79907b47ac82852479"
server_key_version: "encryptionkey@0"
I20260619 20:35:56.672978 25434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "1482af400250477a8996f2fe0beeca41"
format_stamp: "Formatted at 2026-06-19 20:35:56 on dist-test-slave-4m58"
server_key: "702b9cdaeda52e4f729ec13d622fdc92"
server_key_iv: "9d6200a9c96ccd79907b47ac82852479"
server_key_version: "encryptionkey@0"
I20260619 20:35:56.674239 25434 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:35:56.674996 25449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:56.675145 25434 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:35:56.675225 25434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "1482af400250477a8996f2fe0beeca41"
format_stamp: "Formatted at 2026-06-19 20:35:56 on dist-test-slave-4m58"
server_key: "702b9cdaeda52e4f729ec13d622fdc92"
server_key_iv: "9d6200a9c96ccd79907b47ac82852479"
server_key_version: "encryptionkey@0"
I20260619 20:35:56.675284 25434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:56.684626 25434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:56.686458 25434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:56.686584 25434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:56.690246 25434 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35759
I20260619 20:35:56.690294 25501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35759 every 8 connection(s)
I20260619 20:35:56.690608 25434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:35:56.691520 25502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:56.693787 25502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1482af400250477a8996f2fe0beeca41: Bootstrap starting.
I20260619 20:35:56.694347 25502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1482af400250477a8996f2fe0beeca41: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:56.694587 25502 log.cc:824] T 00000000000000000000000000000000 P 1482af400250477a8996f2fe0beeca41: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:56.695276 25502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1482af400250477a8996f2fe0beeca41: No bootstrap required, opened a new log
I20260619 20:35:56.696862 25502 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1482af400250477a8996f2fe0beeca41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1482af400250477a8996f2fe0beeca41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35759 } }
I20260619 20:35:56.697000 25502 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1482af400250477a8996f2fe0beeca41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:56.697021 25502 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1482af400250477a8996f2fe0beeca41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1482af400250477a8996f2fe0beeca41, State: Initialized, Role: FOLLOWER
I20260619 20:35:56.697144 25502 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1482af400250477a8996f2fe0beeca41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1482af400250477a8996f2fe0beeca41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35759 } }
I20260619 20:35:56.697212 25502 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1482af400250477a8996f2fe0beeca41 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:35:56.697234 25502 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1482af400250477a8996f2fe0beeca41 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:35:56.697278 25502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1482af400250477a8996f2fe0beeca41 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:56.697811 25502 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1482af400250477a8996f2fe0beeca41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1482af400250477a8996f2fe0beeca41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35759 } }
I20260619 20:35:56.697902 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 25434
I20260619 20:35:56.697917 25502 leader_election.cc:302] T 00000000000000000000000000000000 P 1482af400250477a8996f2fe0beeca41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1482af400250477a8996f2fe0beeca41; no voters:
I20260619 20:35:56.698022 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:35:56.698073 25502 leader_election.cc:288] T 00000000000000000000000000000000 P 1482af400250477a8996f2fe0beeca41 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:35:56.698130 25505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1482af400250477a8996f2fe0beeca41 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:35:56.698235 4429 external_mini_cluster.cc:1428] Setting key 5a01b6f0c78f046558b4eb174805f6b8
I20260619 20:35:56.698290 25505 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1482af400250477a8996f2fe0beeca41 [term 1 LEADER]: Becoming Leader. State: Replica: 1482af400250477a8996f2fe0beeca41, State: Running, Role: LEADER
I20260619 20:35:56.698369 25502 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1482af400250477a8996f2fe0beeca41 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:35:56.698418 25505 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1482af400250477a8996f2fe0beeca41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1482af400250477a8996f2fe0beeca41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35759 } }
I20260619 20:35:56.698812 25506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1482af400250477a8996f2fe0beeca41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1482af400250477a8996f2fe0beeca41. Latest consensus state: current_term: 1 leader_uuid: "1482af400250477a8996f2fe0beeca41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1482af400250477a8996f2fe0beeca41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35759 } } }
I20260619 20:35:56.698956 25507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1482af400250477a8996f2fe0beeca41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1482af400250477a8996f2fe0beeca41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1482af400250477a8996f2fe0beeca41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35759 } } }
I20260619 20:35:56.699074 25507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1482af400250477a8996f2fe0beeca41 [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:56.698952 25506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1482af400250477a8996f2fe0beeca41 [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:56.699429 25516 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:35:56.700008 25516 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:35:56.701298 25516 catalog_manager.cc:1347] Generated new cluster ID: 6de7c84192404bbea4243b8f17ed73c0
I20260619 20:35:56.701345 25516 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:35:56.704677 25512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:56.698822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:49869 (local address 127.0.0.1:35759)
0619 20:35:56.698963 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:56.698978 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:56.699112 (+ 134us) server_negotiation.cc:407] Connection header received
0619 20:35:56.699287 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:56.699294 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:56.699355 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:56.699495 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:56.699973 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:56.701327 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:56.702479 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:56.703100 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:56.703209 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:56.703988 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:56.703992 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:56.703993 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:35:56.704194 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:56.704238 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:35:56.704252 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:56.704327 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:56.704438 (+ 111us) server_negotiation.cc:299] Negotiation successful
0619 20:35:56.704468 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":70,"threads_started":1}
I20260619 20:35:56.705617 4429 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:35:56.710728 25512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:56.707203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:47377 (local address 127.0.0.1:35759)
0619 20:35:56.707242 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:56.707244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:56.707348 (+ 104us) server_negotiation.cc:407] Connection header received
0619 20:35:56.707426 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:56.707427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:56.707437 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:56.707462 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:56.707627 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:56.708384 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:56.709359 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:56.709675 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:56.709712 (+ 37us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:56.710473 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:56.710475 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:56.710475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:35:56.710479 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:56.710493 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:35:56.710497 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:56.710526 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:56.710633 (+ 107us) server_negotiation.cc:299] Negotiation successful
0619 20:35:56.710647 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:35:56.714140 25516 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:35:56.714901 25516 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:35:56.719345 25516 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1482af400250477a8996f2fe0beeca41: Generated new TSK 0
I20260619 20:35:56.719566 25516 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:35:56.723378 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 25434
W20260619 20:35:56.726979 25419 connection.cc:535] client connection to 127.0.0.1:35759 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35759): Connection reset by peer (error 104)
2026-06-19T20:35:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1265 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:35:56 dist-test-slave-4m58 krb5kdc[25533](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:35:56 dist-test-slave-4m58 krb5kdc[25533](info): set up 2 sockets
Jun 19 20:35:56 dist-test-slave-4m58 krb5kdc[25533](info): commencing operation
krb5kdc: starting...
W20260619 20:35:57.684633 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.943s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:35:57 dist-test-slave-4m58 krb5kdc[25533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:35:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:35:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:35:57.807610 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37503
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:35:57.890316 25546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:57.890545 25546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:57.890569 25546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:57.891772 25546 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:35:57.891822 25546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:57.891860 25546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:57.891875 25546 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:35:57.891898 25546 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:35:57.893442 25546 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:57.893702 25546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:57.893919 25546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:57.898725 25554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:57.898864 25552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:57.898725 25551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:57.899044 25546 server_base.cc:1034] running on GCE node
I20260619 20:35:57.899279 25546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:57.899542 25546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:57.900722 25546 hybrid_clock.cc:648] HybridClock initialized: now 1781901357900705 us; error 31 us; skew 500 ppm
Jun 19 20:35:57 dist-test-slave-4m58 krb5kdc[25533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901357, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:35:57.903949 25546 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:35:57.904412 25546 webserver.cc:460] Webserver started at http://127.0.0.1:39567/ using document root <none> and password file <none>
I20260619 20:35:57.904647 25546 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:57.904697 25546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:57.904826 25546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:57.905872 25546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "226c97d9787e4df8a336ce7024376808"
format_stamp: "Formatted at 2026-06-19 20:35:57 on dist-test-slave-4m58"
server_key: "4933c39f606e1fbdd55577643aa17a78"
server_key_iv: "5dd2e476615e811aa3e1e11f76b621d5"
server_key_version: "encryptionkey@0"
I20260619 20:35:57.906213 25546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "226c97d9787e4df8a336ce7024376808"
format_stamp: "Formatted at 2026-06-19 20:35:57 on dist-test-slave-4m58"
server_key: "4933c39f606e1fbdd55577643aa17a78"
server_key_iv: "5dd2e476615e811aa3e1e11f76b621d5"
server_key_version: "encryptionkey@0"
I20260619 20:35:57.907454 25546 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:35:57.908138 25561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:57.908293 25546 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:57.908373 25546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "226c97d9787e4df8a336ce7024376808"
format_stamp: "Formatted at 2026-06-19 20:35:57 on dist-test-slave-4m58"
server_key: "4933c39f606e1fbdd55577643aa17a78"
server_key_iv: "5dd2e476615e811aa3e1e11f76b621d5"
server_key_version: "encryptionkey@0"
I20260619 20:35:57.908442 25546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:57.911639 25546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:57.914080 25546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:57.914248 25546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:57.918020 25546 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37503
I20260619 20:35:57.918061 25613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37503 every 8 connection(s)
I20260619 20:35:57.918373 25546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:35:57.919284 25614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:57.921890 25614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 226c97d9787e4df8a336ce7024376808: Bootstrap starting.
I20260619 20:35:57.922490 25614 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 226c97d9787e4df8a336ce7024376808: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:57.922752 25614 log.cc:824] T 00000000000000000000000000000000 P 226c97d9787e4df8a336ce7024376808: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:57.923487 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 25546
I20260619 20:35:57.923523 25614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 226c97d9787e4df8a336ce7024376808: No bootstrap required, opened a new log
I20260619 20:35:57.923622 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:35:57.923816 4429 external_mini_cluster.cc:1428] Setting key 6319e9b54a443597ff7f5d4e108b5052
I20260619 20:35:57.925482 25614 raft_consensus.cc:348] T 00000000000000000000000000000000 P 226c97d9787e4df8a336ce7024376808 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "226c97d9787e4df8a336ce7024376808" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37503 } }
I20260619 20:35:57.925650 25614 raft_consensus.cc:374] T 00000000000000000000000000000000 P 226c97d9787e4df8a336ce7024376808 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:57.925690 25614 raft_consensus.cc:729] T 00000000000000000000000000000000 P 226c97d9787e4df8a336ce7024376808 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 226c97d9787e4df8a336ce7024376808, State: Initialized, Role: FOLLOWER
I20260619 20:35:57.925879 25614 consensus_queue.cc:260] T 00000000000000000000000000000000 P 226c97d9787e4df8a336ce7024376808 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "226c97d9787e4df8a336ce7024376808" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37503 } }
I20260619 20:35:57.925982 25614 raft_consensus.cc:388] T 00000000000000000000000000000000 P 226c97d9787e4df8a336ce7024376808 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:35:57.926023 25614 raft_consensus.cc:482] T 00000000000000000000000000000000 P 226c97d9787e4df8a336ce7024376808 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:35:57.926071 25614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 226c97d9787e4df8a336ce7024376808 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:57.926743 25618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:57.924400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:60217 (local address 127.0.0.1:37503)
0619 20:35:57.924571 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:57.924580 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:57.924607 (+ 27us) server_negotiation.cc:407] Connection header received
0619 20:35:57.925457 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:57.925464 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:57.925493 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:57.925613 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:57.926329 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:57.926332 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:57.926333 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:35:57.926337 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:57.926396 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:35:57.926408 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:57.926466 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:57.926525 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 20:35:57.926567 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":71,"threads_started":1}
I20260619 20:35:57.927109 25614 raft_consensus.cc:504] T 00000000000000000000000000000000 P 226c97d9787e4df8a336ce7024376808 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "226c97d9787e4df8a336ce7024376808" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37503 } }
I20260619 20:35:57.927213 25614 leader_election.cc:302] T 00000000000000000000000000000000 P 226c97d9787e4df8a336ce7024376808 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 226c97d9787e4df8a336ce7024376808; no voters:
I20260619 20:35:57.927383 25614 leader_election.cc:288] T 00000000000000000000000000000000 P 226c97d9787e4df8a336ce7024376808 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:35:57.927425 25619 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 226c97d9787e4df8a336ce7024376808 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:35:57.927533 25619 raft_consensus.cc:686] T 00000000000000000000000000000000 P 226c97d9787e4df8a336ce7024376808 [term 1 LEADER]: Becoming Leader. State: Replica: 226c97d9787e4df8a336ce7024376808, State: Running, Role: LEADER
I20260619 20:35:57.927600 4429 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:35:57.927659 25614 sys_catalog.cc:564] T 00000000000000000000000000000000 P 226c97d9787e4df8a336ce7024376808 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:35:57.928805 25619 consensus_queue.cc:237] T 00000000000000000000000000000000 P 226c97d9787e4df8a336ce7024376808 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "226c97d9787e4df8a336ce7024376808" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37503 } }
I20260619 20:35:57.929467 25621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 226c97d9787e4df8a336ce7024376808 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 226c97d9787e4df8a336ce7024376808. Latest consensus state: current_term: 1 leader_uuid: "226c97d9787e4df8a336ce7024376808" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "226c97d9787e4df8a336ce7024376808" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37503 } } }
I20260619 20:35:57.929561 25621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 226c97d9787e4df8a336ce7024376808 [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:57.929745 25620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 226c97d9787e4df8a336ce7024376808 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "226c97d9787e4df8a336ce7024376808" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "226c97d9787e4df8a336ce7024376808" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37503 } } }
I20260619 20:35:57.929811 25620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 226c97d9787e4df8a336ce7024376808 [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:57.929868 25633 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:35:57.930327 25618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:57.928917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:51957 (local address 127.0.0.1:37503)
0619 20:35:57.928953 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:57.928954 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:57.929080 (+ 126us) server_negotiation.cc:407] Connection header received
0619 20:35:57.929132 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:57.929133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:57.929139 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:57.929204 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:57.929986 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:57.929987 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:57.929988 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:35:57.929992 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:57.930011 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:35:57.930016 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:57.930082 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:57.930254 (+ 172us) server_negotiation.cc:299] Negotiation successful
0619 20:35:57.930268 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:35:57.930595 25633 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:35:57.932133 25633 catalog_manager.cc:1347] Generated new cluster ID: d68b739290324f6795e31749d57de87a
I20260619 20:35:57.932209 25633 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:35:57.938062 25633 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:35:57.938325 25633 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:35:57.942353 25633 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 226c97d9787e4df8a336ce7024376808: Generated new TSK 0
I20260619 20:35:57.942553 25633 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:35:57.948596 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 25546
2026-06-19T20:35:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1224 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:35:58 dist-test-slave-4m58 krb5kdc[25645](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:35:58 dist-test-slave-4m58 krb5kdc[25645](info): set up 2 sockets
Jun 19 20:35:58 dist-test-slave-4m58 krb5kdc[25645](info): commencing operation
krb5kdc: starting...
W20260619 20:35:58.908705 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.943s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:35:58 dist-test-slave-4m58 krb5kdc[25645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:35:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:35:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:35:59.027405 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41111
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:35:59.109912 25658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:35:59.110157 25658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:35:59.110188 25658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:35:59.111377 25658 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:35:59.111433 25658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:35:59.111451 25658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:35:59.111466 25658 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:35:59.111480 25658 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:35:59.113075 25658 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41111
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:35:59.113346 25658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:35:59.113564 25658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:35:59.119626 25663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:59.119593 25664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:35:59.119596 25666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:35:59.119684 25658 server_base.cc:1034] running on GCE node
I20260619 20:35:59.120164 25658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:35:59.120517 25658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:35:59.121744 25658 hybrid_clock.cc:648] HybridClock initialized: now 1781901359121727 us; error 43 us; skew 500 ppm
Jun 19 20:35:59 dist-test-slave-4m58 krb5kdc[25645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901359, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:35:59.127970 25658 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:35:59.128499 25658 webserver.cc:460] Webserver started at http://127.0.0.1:36861/ using document root <none> and password file <none>
I20260619 20:35:59.128734 25658 fs_manager.cc:359] Metadata directory not provided
I20260619 20:35:59.128796 25658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:35:59.128930 25658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:35:59.130008 25658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "8ef386d1b2114c61851fc7abff9dd81a"
format_stamp: "Formatted at 2026-06-19 20:35:59 on dist-test-slave-4m58"
server_key: "43feb4b2ce251f9bec160b681757a866"
server_key_iv: "c8b35070af8147ead7bae10a37dd42f8"
server_key_version: "encryptionkey@0"
I20260619 20:35:59.130343 25658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "8ef386d1b2114c61851fc7abff9dd81a"
format_stamp: "Formatted at 2026-06-19 20:35:59 on dist-test-slave-4m58"
server_key: "43feb4b2ce251f9bec160b681757a866"
server_key_iv: "c8b35070af8147ead7bae10a37dd42f8"
server_key_version: "encryptionkey@0"
I20260619 20:35:59.131875 25658 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:35:59.132675 25673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:35:59.132901 25658 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:35:59.132997 25658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "8ef386d1b2114c61851fc7abff9dd81a"
format_stamp: "Formatted at 2026-06-19 20:35:59 on dist-test-slave-4m58"
server_key: "43feb4b2ce251f9bec160b681757a866"
server_key_iv: "c8b35070af8147ead7bae10a37dd42f8"
server_key_version: "encryptionkey@0"
I20260619 20:35:59.133065 25658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:35:59.140775 25658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:35:59.143266 25658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:35:59.143404 25658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:35:59.147786 25658 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41111
I20260619 20:35:59.147964 25725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41111 every 8 connection(s)
I20260619 20:35:59.148237 25658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:35:59.149307 25726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:35:59.152278 25726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ef386d1b2114c61851fc7abff9dd81a: Bootstrap starting.
I20260619 20:35:59.152864 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 25658
I20260619 20:35:59.153002 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:35:59.153155 25726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ef386d1b2114c61851fc7abff9dd81a: Neither blocks nor log segments found. Creating new log.
I20260619 20:35:59.153216 4429 external_mini_cluster.cc:1428] Setting key 69d49e98e40f35b1c63c21423d7d824c
I20260619 20:35:59.153522 25726 log.cc:824] T 00000000000000000000000000000000 P 8ef386d1b2114c61851fc7abff9dd81a: Log is configured to *not* fsync() on all Append() calls
I20260619 20:35:59.154815 25726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ef386d1b2114c61851fc7abff9dd81a: No bootstrap required, opened a new log
I20260619 20:35:59.157546 25726 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ef386d1b2114c61851fc7abff9dd81a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef386d1b2114c61851fc7abff9dd81a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41111 } }
I20260619 20:35:59.157802 25726 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ef386d1b2114c61851fc7abff9dd81a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:35:59.157843 25726 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ef386d1b2114c61851fc7abff9dd81a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef386d1b2114c61851fc7abff9dd81a, State: Initialized, Role: FOLLOWER
I20260619 20:35:59.158011 25726 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ef386d1b2114c61851fc7abff9dd81a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef386d1b2114c61851fc7abff9dd81a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41111 } }
I20260619 20:35:59.158106 25726 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ef386d1b2114c61851fc7abff9dd81a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:35:59.158138 25726 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ef386d1b2114c61851fc7abff9dd81a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:35:59.158190 25726 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ef386d1b2114c61851fc7abff9dd81a [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:35:59.159162 25726 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ef386d1b2114c61851fc7abff9dd81a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef386d1b2114c61851fc7abff9dd81a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41111 } }
I20260619 20:35:59.159304 25726 leader_election.cc:302] T 00000000000000000000000000000000 P 8ef386d1b2114c61851fc7abff9dd81a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ef386d1b2114c61851fc7abff9dd81a; no voters:
I20260619 20:35:59.159516 25726 leader_election.cc:288] T 00000000000000000000000000000000 P 8ef386d1b2114c61851fc7abff9dd81a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:35:59.159547 25731 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ef386d1b2114c61851fc7abff9dd81a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:35:59.159682 25731 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ef386d1b2114c61851fc7abff9dd81a [term 1 LEADER]: Becoming Leader. State: Replica: 8ef386d1b2114c61851fc7abff9dd81a, State: Running, Role: LEADER
I20260619 20:35:59.159787 25731 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ef386d1b2114c61851fc7abff9dd81a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef386d1b2114c61851fc7abff9dd81a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41111 } }
I20260619 20:35:59.159955 25726 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ef386d1b2114c61851fc7abff9dd81a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:35:59.160480 25732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ef386d1b2114c61851fc7abff9dd81a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ef386d1b2114c61851fc7abff9dd81a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef386d1b2114c61851fc7abff9dd81a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41111 } } }
I20260619 20:35:59.160670 25732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ef386d1b2114c61851fc7abff9dd81a [sys.catalog]: This master's current role is: LEADER
I20260619 20:35:59.160458 25733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ef386d1b2114c61851fc7abff9dd81a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ef386d1b2114c61851fc7abff9dd81a. Latest consensus state: current_term: 1 leader_uuid: "8ef386d1b2114c61851fc7abff9dd81a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef386d1b2114c61851fc7abff9dd81a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41111 } } }
I20260619 20:35:59.160975 25733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ef386d1b2114c61851fc7abff9dd81a [sys.catalog]: This master's current role is: LEADER
Jun 19 20:35:59 dist-test-slave-4m58 krb5kdc[25645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
W20260619 20:35:59.162092 25746 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8ef386d1b2114c61851fc7abff9dd81a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:35:59.162784 25746 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:35:59.162863 25740 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:35:59.163120 25740 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:35:59.164745 25740 catalog_manager.cc:1347] Generated new cluster ID: e3c6c32319384c438e0a8296b27550d3
I20260619 20:35:59.164801 25740 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:35:59.167582 25730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:59.153961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:47753 (local address 127.0.0.1:41111)
0619 20:35:59.154252 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:59.154266 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:59.154324 (+ 58us) server_negotiation.cc:407] Connection header received
0619 20:35:59.155010 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:59.155019 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:59.155083 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:59.155239 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:59.156195 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:59.157150 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:59.158324 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:59.159012 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:59.159148 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:59.162462 (+ 3314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:59.162468 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:59.162468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:35:59.162484 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:59.164776 (+ 2292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:59.165461 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:59.165478 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:59.165479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:59.165628 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:59.165917 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:59.165918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:59.165919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:59.166348 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:59.166561 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:59.167150 (+ 589us) server_negotiation.cc:299] Negotiation successful
0619 20:35:59.167294 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":92,"threads_started":1}
I20260619 20:35:59.168951 4429 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:35:59.186914 25740 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:35:59.187798 25740 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:35:59.189455 25730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:35:59.180378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:58451 (local address 127.0.0.1:41111)
0619 20:35:59.180445 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 20:35:59.180448 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:35:59.180538 (+ 90us) server_negotiation.cc:407] Connection header received
0619 20:35:59.180648 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:35:59.180650 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:35:59.180662 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:35:59.180724 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:35:59.181731 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:59.182907 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:59.184352 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:35:59.184691 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:35:59.184777 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:35:59.186587 (+ 1810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:35:59.186589 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:35:59.186589 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:35:59.186598 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:35:59.188268 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:59.188678 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:59.188679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:59.188680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:59.188738 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:35:59.188933 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:35:59.188934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:35:59.188934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:35:59.189056 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:35:59.189174 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 20:35:59.189285 (+ 111us) server_negotiation.cc:299] Negotiation successful
0619 20:35:59.189357 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 20:35:59.192461 25740 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ef386d1b2114c61851fc7abff9dd81a: Generated new TSK 0
I20260619 20:35:59.192780 25740 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:35:59.201258 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 25658
W20260619 20:35:59.205973 25643 connection.cc:535] client connection to 127.0.0.1:41111 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41111): Connection reset by peer (error 104)
2026-06-19T20:35:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1254 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:35:59 dist-test-slave-4m58 krb5kdc[25757](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:35:59 dist-test-slave-4m58 krb5kdc[25757](info): set up 2 sockets
Jun 19 20:35:59 dist-test-slave-4m58 krb5kdc[25757](info): commencing operation
krb5kdc: starting...
W20260619 20:36:00.168692 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.949s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:36:00 dist-test-slave-4m58 krb5kdc[25757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:36:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:36:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:36:00.297003 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38669
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:00.376549 25770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:00.376787 25770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:00.376812 25770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:00.378039 25770 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:00.378096 25770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:00.378109 25770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:00.378120 25770 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:00.378131 25770 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:00.379664 25770 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38669
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:00.379945 25770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:00.380172 25770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:00.386242 25775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:00.386461 25770 server_base.cc:1034] running on GCE node
W20260619 20:36:00.386411 25776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:00.386615 25778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:00.386967 25770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:00.387356 25770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:00.388607 25770 hybrid_clock.cc:648] HybridClock initialized: now 1781901360388563 us; error 59 us; skew 500 ppm
Jun 19 20:36:00 dist-test-slave-4m58 krb5kdc[25757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901360, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:00.391722 25770 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:36:00.392318 25770 webserver.cc:460] Webserver started at http://127.0.0.1:41699/ using document root <none> and password file <none>
I20260619 20:36:00.392550 25770 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:00.392601 25770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:00.392701 25770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:00.393765 25770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "c13ba838daa74edcb0163f4de0b656fb"
format_stamp: "Formatted at 2026-06-19 20:36:00 on dist-test-slave-4m58"
server_key: "247154e049d3a93f02bfa476740e8baa"
server_key_iv: "8dcd423ac63f21ef7517d265bd9e5715"
server_key_version: "encryptionkey@0"
I20260619 20:36:00.394063 25770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "c13ba838daa74edcb0163f4de0b656fb"
format_stamp: "Formatted at 2026-06-19 20:36:00 on dist-test-slave-4m58"
server_key: "247154e049d3a93f02bfa476740e8baa"
server_key_iv: "8dcd423ac63f21ef7517d265bd9e5715"
server_key_version: "encryptionkey@0"
I20260619 20:36:00.395583 25770 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:36:00.396688 25785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:00.396945 25770 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:00.397050 25770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "c13ba838daa74edcb0163f4de0b656fb"
format_stamp: "Formatted at 2026-06-19 20:36:00 on dist-test-slave-4m58"
server_key: "247154e049d3a93f02bfa476740e8baa"
server_key_iv: "8dcd423ac63f21ef7517d265bd9e5715"
server_key_version: "encryptionkey@0"
I20260619 20:36:00.397122 25770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:00.415781 25770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:00.418141 25770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:00.418269 25770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:00.422775 25770 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38669
I20260619 20:36:00.422869 25837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38669 every 8 connection(s)
I20260619 20:36:00.423219 25770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:36:00.424369 25838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:00.427307 25838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c13ba838daa74edcb0163f4de0b656fb: Bootstrap starting.
I20260619 20:36:00.428112 25838 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c13ba838daa74edcb0163f4de0b656fb: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:00.428424 25838 log.cc:824] T 00000000000000000000000000000000 P c13ba838daa74edcb0163f4de0b656fb: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:00.429420 25838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c13ba838daa74edcb0163f4de0b656fb: No bootstrap required, opened a new log
I20260619 20:36:00.431746 25838 raft_consensus.cc:348] T 00000000000000000000000000000000 P c13ba838daa74edcb0163f4de0b656fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13ba838daa74edcb0163f4de0b656fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38669 } }
I20260619 20:36:00.432126 25838 raft_consensus.cc:374] T 00000000000000000000000000000000 P c13ba838daa74edcb0163f4de0b656fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:00.432124 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 25770
I20260619 20:36:00.432178 25838 raft_consensus.cc:729] T 00000000000000000000000000000000 P c13ba838daa74edcb0163f4de0b656fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c13ba838daa74edcb0163f4de0b656fb, State: Initialized, Role: FOLLOWER
I20260619 20:36:00.432258 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:36:00.432482 4429 external_mini_cluster.cc:1428] Setting key 0e5b7eca63f9831528958e5c5e24a180
I20260619 20:36:00.432469 25838 consensus_queue.cc:260] T 00000000000000000000000000000000 P c13ba838daa74edcb0163f4de0b656fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13ba838daa74edcb0163f4de0b656fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38669 } }
I20260619 20:36:00.432641 25838 raft_consensus.cc:388] T 00000000000000000000000000000000 P c13ba838daa74edcb0163f4de0b656fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:36:00.432691 25838 raft_consensus.cc:482] T 00000000000000000000000000000000 P c13ba838daa74edcb0163f4de0b656fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:36:00.432752 25838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c13ba838daa74edcb0163f4de0b656fb [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:00.434371 25838 raft_consensus.cc:504] T 00000000000000000000000000000000 P c13ba838daa74edcb0163f4de0b656fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13ba838daa74edcb0163f4de0b656fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38669 } }
I20260619 20:36:00.434561 25838 leader_election.cc:302] T 00000000000000000000000000000000 P c13ba838daa74edcb0163f4de0b656fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c13ba838daa74edcb0163f4de0b656fb; no voters:
I20260619 20:36:00.434778 25838 leader_election.cc:288] T 00000000000000000000000000000000 P c13ba838daa74edcb0163f4de0b656fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:36:00.434861 25843 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c13ba838daa74edcb0163f4de0b656fb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:00.435101 25843 raft_consensus.cc:686] T 00000000000000000000000000000000 P c13ba838daa74edcb0163f4de0b656fb [term 1 LEADER]: Becoming Leader. State: Replica: c13ba838daa74edcb0163f4de0b656fb, State: Running, Role: LEADER
I20260619 20:36:00.435168 25838 sys_catalog.cc:564] T 00000000000000000000000000000000 P c13ba838daa74edcb0163f4de0b656fb [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:00.435333 25843 consensus_queue.cc:237] T 00000000000000000000000000000000 P c13ba838daa74edcb0163f4de0b656fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13ba838daa74edcb0163f4de0b656fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38669 } }
I20260619 20:36:00.435933 25844 sys_catalog.cc:455] T 00000000000000000000000000000000 P c13ba838daa74edcb0163f4de0b656fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c13ba838daa74edcb0163f4de0b656fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13ba838daa74edcb0163f4de0b656fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38669 } } }
I20260619 20:36:00.436058 25844 sys_catalog.cc:458] T 00000000000000000000000000000000 P c13ba838daa74edcb0163f4de0b656fb [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:00.436319 25845 sys_catalog.cc:455] T 00000000000000000000000000000000 P c13ba838daa74edcb0163f4de0b656fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader c13ba838daa74edcb0163f4de0b656fb. Latest consensus state: current_term: 1 leader_uuid: "c13ba838daa74edcb0163f4de0b656fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13ba838daa74edcb0163f4de0b656fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38669 } } }
I20260619 20:36:00.436394 25845 sys_catalog.cc:458] T 00000000000000000000000000000000 P c13ba838daa74edcb0163f4de0b656fb [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:00.436444 25852 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:36:00.437222 25852 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:36:00.440313 25842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:00.433211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:48357 (local address 127.0.0.1:38669)
0619 20:36:00.433534 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:00.433549 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:00.433612 (+ 63us) server_negotiation.cc:407] Connection header received
0619 20:36:00.433825 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:00.433835 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:00.433928 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:00.434143 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:00.434426 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:00.435986 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:00.437295 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:00.438075 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:00.438216 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:00.439258 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:00.439263 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:00.439264 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:36:00.439537 (+ 273us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:00.439592 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:36:00.439612 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:00.439710 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:00.440033 (+ 323us) server_negotiation.cc:299] Negotiation successful
0619 20:36:00.440069 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":151,"threads_started":1}
I20260619 20:36:00.440667 25852 catalog_manager.cc:1347] Generated new cluster ID: bed415a28b83475f994e8d451a19fde3
I20260619 20:36:00.440752 25852 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:36:00.445096 4429 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:36:00.456213 25842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:00.446796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:60955 (local address 127.0.0.1:38669)
0619 20:36:00.446838 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:00.446840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:00.452142 (+ 5302us) server_negotiation.cc:407] Connection header received
0619 20:36:00.452210 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:00.452212 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:00.452233 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:00.452330 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:00.452531 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:00.453356 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:00.454478 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:00.454825 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:00.454891 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:00.455775 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:00.455778 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:00.455778 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:36:00.455785 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:00.455810 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:36:00.455816 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:00.455897 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:00.456080 (+ 183us) server_negotiation.cc:299] Negotiation successful
0619 20:36:00.456105 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:36:00.461364 25852 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:36:00.461962 25852 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:36:00.470414 25852 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c13ba838daa74edcb0163f4de0b656fb: Generated new TSK 0
I20260619 20:36:00.470717 25852 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:36:00.477368 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 25770
W20260619 20:36:00.482236 25755 connection.cc:535] client connection to 127.0.0.1:38669 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38669): Connection reset by peer (error 104)
2026-06-19T20:36:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1276 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:36:00 dist-test-slave-4m58 krb5kdc[25869](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:36:00 dist-test-slave-4m58 krb5kdc[25869](info): set up 2 sockets
Jun 19 20:36:00 dist-test-slave-4m58 krb5kdc[25869](info): commencing operation
krb5kdc: starting...
W20260619 20:36:01.453297 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.957s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:36:01 dist-test-slave-4m58 krb5kdc[25869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:36:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:36:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:36:01.575291 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38493
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:01.655499 25882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:01.655730 25882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:01.655753 25882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:01.657125 25882 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:01.657195 25882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:01.657209 25882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:01.657323 25882 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:01.657351 25882 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:01.659201 25882 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38493
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901102012824-4429-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:01.659438 25882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:01.659627 25882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:01.664357 25888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:01.664367 25887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:01.664381 25890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:01.664534 25882 server_base.cc:1034] running on GCE node
I20260619 20:36:01.664803 25882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:01.665107 25882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:01.666278 25882 hybrid_clock.cc:648] HybridClock initialized: now 1781901361666270 us; error 34 us; skew 500 ppm
Jun 19 20:36:01 dist-test-slave-4m58 krb5kdc[25869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901361, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:01.669128 25882 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:36:01.669585 25882 webserver.cc:460] Webserver started at http://127.0.0.1:43029/ using document root <none> and password file <none>
I20260619 20:36:01.669816 25882 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:01.669865 25882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:01.669989 25882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:01.671054 25882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "e5f6ed01244b48739f4d3d90c029006b"
format_stamp: "Formatted at 2026-06-19 20:36:01 on dist-test-slave-4m58"
server_key: "cc74060d2762668d8b616ec78f28a5fe"
server_key_iv: "40c240acf5dbc663b7f9389728e4eecf"
server_key_version: "encryptionkey@0"
I20260619 20:36:01.671402 25882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "e5f6ed01244b48739f4d3d90c029006b"
format_stamp: "Formatted at 2026-06-19 20:36:01 on dist-test-slave-4m58"
server_key: "cc74060d2762668d8b616ec78f28a5fe"
server_key_iv: "40c240acf5dbc663b7f9389728e4eecf"
server_key_version: "encryptionkey@0"
I20260619 20:36:01.672812 25882 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:36:01.673677 25897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:01.673919 25882 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:36:01.674026 25882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "e5f6ed01244b48739f4d3d90c029006b"
format_stamp: "Formatted at 2026-06-19 20:36:01 on dist-test-slave-4m58"
server_key: "cc74060d2762668d8b616ec78f28a5fe"
server_key_iv: "40c240acf5dbc663b7f9389728e4eecf"
server_key_version: "encryptionkey@0"
I20260619 20:36:01.674103 25882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:01.677510 25882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:01.679587 25882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:01.679728 25882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:01.683763 25882 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38493
I20260619 20:36:01.683822 25949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38493 every 8 connection(s)
I20260619 20:36:01.684298 25882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:36:01.685354 25950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:01.687599 25950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5f6ed01244b48739f4d3d90c029006b: Bootstrap starting.
I20260619 20:36:01.688313 25950 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5f6ed01244b48739f4d3d90c029006b: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:01.688562 25950 log.cc:824] T 00000000000000000000000000000000 P e5f6ed01244b48739f4d3d90c029006b: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:01.689352 25950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5f6ed01244b48739f4d3d90c029006b: No bootstrap required, opened a new log
I20260619 20:36:01.690188 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 25882
I20260619 20:36:01.690312 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:36:01.690482 4429 external_mini_cluster.cc:1428] Setting key e65e2c270d484ca7a14b44eda5028fd4
I20260619 20:36:01.691493 25950 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5f6ed01244b48739f4d3d90c029006b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5f6ed01244b48739f4d3d90c029006b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38493 } }
I20260619 20:36:01.691663 25950 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5f6ed01244b48739f4d3d90c029006b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:01.691686 25950 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5f6ed01244b48739f4d3d90c029006b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5f6ed01244b48739f4d3d90c029006b, State: Initialized, Role: FOLLOWER
I20260619 20:36:01.691818 25950 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5f6ed01244b48739f4d3d90c029006b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5f6ed01244b48739f4d3d90c029006b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38493 } }
I20260619 20:36:01.691931 25950 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5f6ed01244b48739f4d3d90c029006b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:36:01.691963 25950 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5f6ed01244b48739f4d3d90c029006b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:36:01.692317 25950 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5f6ed01244b48739f4d3d90c029006b [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:01.692835 25954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:01.690984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:35593 (local address 127.0.0.1:38493)
0619 20:36:01.691189 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:01.691196 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:01.691223 (+ 27us) server_negotiation.cc:407] Connection header received
0619 20:36:01.691499 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:01.691507 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:01.691540 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:01.691711 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:01.692419 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:01.692422 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:01.692423 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:36:01.692427 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:01.692472 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:36:01.692486 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:01.692542 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:01.692606 (+ 64us) server_negotiation.cc:299] Negotiation successful
0619 20:36:01.692648 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":89,"threads_started":1}
I20260619 20:36:01.693482 25950 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5f6ed01244b48739f4d3d90c029006b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5f6ed01244b48739f4d3d90c029006b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38493 } }
I20260619 20:36:01.693599 25950 leader_election.cc:302] T 00000000000000000000000000000000 P e5f6ed01244b48739f4d3d90c029006b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5f6ed01244b48739f4d3d90c029006b; no voters:
I20260619 20:36:01.693704 4429 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:36:01.693760 25950 leader_election.cc:288] T 00000000000000000000000000000000 P e5f6ed01244b48739f4d3d90c029006b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:36:01.693814 25955 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5f6ed01244b48739f4d3d90c029006b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:01.693946 25955 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5f6ed01244b48739f4d3d90c029006b [term 1 LEADER]: Becoming Leader. State: Replica: e5f6ed01244b48739f4d3d90c029006b, State: Running, Role: LEADER
I20260619 20:36:01.693996 25950 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5f6ed01244b48739f4d3d90c029006b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:01.694106 25955 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5f6ed01244b48739f4d3d90c029006b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5f6ed01244b48739f4d3d90c029006b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38493 } }
I20260619 20:36:01.694447 25955 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5f6ed01244b48739f4d3d90c029006b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5f6ed01244b48739f4d3d90c029006b. Latest consensus state: current_term: 1 leader_uuid: "e5f6ed01244b48739f4d3d90c029006b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5f6ed01244b48739f4d3d90c029006b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38493 } } }
I20260619 20:36:01.694651 25955 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5f6ed01244b48739f4d3d90c029006b [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:01.694585 25956 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5f6ed01244b48739f4d3d90c029006b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5f6ed01244b48739f4d3d90c029006b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5f6ed01244b48739f4d3d90c029006b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38493 } } }
I20260619 20:36:01.695024 25956 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5f6ed01244b48739f4d3d90c029006b [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:01.695281 25966 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:36:01.695901 25966 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:36:01.696405 25954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:01.695232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:59803 (local address 127.0.0.1:38493)
0619 20:36:01.695270 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:01.695272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:01.695400 (+ 128us) server_negotiation.cc:407] Connection header received
0619 20:36:01.695501 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:01.695501 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:01.695507 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:01.695567 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:01.696193 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:01.696194 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:01.696194 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:36:01.696198 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:01.696221 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:36:01.696225 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:01.696286 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:01.696333 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 20:36:01.696347 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:36:01.697391 25966 catalog_manager.cc:1347] Generated new cluster ID: 51993c4ed1194a39aa00f0423dc4762b
I20260619 20:36:01.697439 25966 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:36:01.700842 25966 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:36:01.701006 25966 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:36:01.710238 25966 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5f6ed01244b48739f4d3d90c029006b: Generated new TSK 0
I20260619 20:36:01.710458 25966 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:36:01.715426 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 25882
2026-06-19T20:36:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1236 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:36:01 dist-test-slave-4m58 krb5kdc[25981](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:36:01 dist-test-slave-4m58 krb5kdc[25981](info): set up 2 sockets
Jun 19 20:36:01 dist-test-slave-4m58 krb5kdc[25981](info): commencing operation
krb5kdc: starting...
W20260619 20:36:02.676652 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.944s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:36:02 dist-test-slave-4m58 krb5kdc[25981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:36:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:36:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
I20260619 20:36:02.795059 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:41139
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:44865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:41139
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:02.877492 25994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:02.878021 25994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:02.878134 25994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:02.881368 25994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:02.881523 25994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:02.881600 25994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:02.881628 25994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:02.881654 25994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:02.884999 25994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:44865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:41139
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:41139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:02.885344 25994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:02.885577 25994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:02.891495 25999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:02.891497 26000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:02.891497 26002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:02.891781 25994 server_base.cc:1034] running on GCE node
I20260619 20:36:02.892071 25994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:02.892382 25994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:02.893558 25994 hybrid_clock.cc:648] HybridClock initialized: now 1781901362893497 us; error 81 us; skew 500 ppm
Jun 19 20:36:02 dist-test-slave-4m58 krb5kdc[25981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901362, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:02.896229 25994 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:36:02.896869 25994 webserver.cc:460] Webserver started at http://127.4.83.126:34409/ using document root <none> and password file <none>
I20260619 20:36:02.897104 25994 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:02.897152 25994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:02.897264 25994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:02.898386 25994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "0a2d77aaeb8f450592d678052d4a0038"
format_stamp: "Formatted at 2026-06-19 20:36:02 on dist-test-slave-4m58"
server_key: "e9487538d8e2c860d2104d7b31241a56"
server_key_iv: "9c5eecaaeea372194f61c184f2e26939"
server_key_version: "encryptionkey@0"
I20260619 20:36:02.898744 25994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "0a2d77aaeb8f450592d678052d4a0038"
format_stamp: "Formatted at 2026-06-19 20:36:02 on dist-test-slave-4m58"
server_key: "e9487538d8e2c860d2104d7b31241a56"
server_key_iv: "9c5eecaaeea372194f61c184f2e26939"
server_key_version: "encryptionkey@0"
I20260619 20:36:02.900231 25994 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 20:36:02.901386 26009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:02.901746 25994 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:02.901850 25994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "0a2d77aaeb8f450592d678052d4a0038"
format_stamp: "Formatted at 2026-06-19 20:36:02 on dist-test-slave-4m58"
server_key: "e9487538d8e2c860d2104d7b31241a56"
server_key_iv: "9c5eecaaeea372194f61c184f2e26939"
server_key_version: "encryptionkey@0"
I20260619 20:36:02.901939 25994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:02.910619 25994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:02.912917 25994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:02.913064 25994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:02.917553 25994 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:41139
I20260619 20:36:02.917743 26061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:41139 every 8 connection(s)
I20260619 20:36:02.917990 25994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:36:02.919307 26062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:02.920399 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 25994
I20260619 20:36:02.920511 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:36:02.920686 4429 external_mini_cluster.cc:1428] Setting key c3625f12f2c8e24af83a67511b0e307c
I20260619 20:36:02.922078 26062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a2d77aaeb8f450592d678052d4a0038: Bootstrap starting.
I20260619 20:36:02.922660 26062 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a2d77aaeb8f450592d678052d4a0038: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:02.922915 26062 log.cc:824] T 00000000000000000000000000000000 P 0a2d77aaeb8f450592d678052d4a0038: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:02.923645 26062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a2d77aaeb8f450592d678052d4a0038: No bootstrap required, opened a new log
I20260619 20:36:02.925906 26062 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a2d77aaeb8f450592d678052d4a0038 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a2d77aaeb8f450592d678052d4a0038" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 41139 } }
I20260619 20:36:02.926095 26062 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a2d77aaeb8f450592d678052d4a0038 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:02.926136 26062 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a2d77aaeb8f450592d678052d4a0038 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a2d77aaeb8f450592d678052d4a0038, State: Initialized, Role: FOLLOWER
I20260619 20:36:02.926333 26062 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a2d77aaeb8f450592d678052d4a0038 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a2d77aaeb8f450592d678052d4a0038" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 41139 } }
I20260619 20:36:02.926416 26062 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a2d77aaeb8f450592d678052d4a0038 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:36:02.926438 26062 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a2d77aaeb8f450592d678052d4a0038 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:36:02.926483 26062 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a2d77aaeb8f450592d678052d4a0038 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:02.927059 26062 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a2d77aaeb8f450592d678052d4a0038 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a2d77aaeb8f450592d678052d4a0038" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 41139 } }
I20260619 20:36:02.927148 26062 leader_election.cc:302] T 00000000000000000000000000000000 P 0a2d77aaeb8f450592d678052d4a0038 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a2d77aaeb8f450592d678052d4a0038; no voters:
I20260619 20:36:02.927320 26062 leader_election.cc:288] T 00000000000000000000000000000000 P 0a2d77aaeb8f450592d678052d4a0038 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:36:02.927354 26067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a2d77aaeb8f450592d678052d4a0038 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:02.927508 26067 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a2d77aaeb8f450592d678052d4a0038 [term 1 LEADER]: Becoming Leader. State: Replica: 0a2d77aaeb8f450592d678052d4a0038, State: Running, Role: LEADER
I20260619 20:36:02.927595 26062 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a2d77aaeb8f450592d678052d4a0038 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:02.927678 26067 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a2d77aaeb8f450592d678052d4a0038 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a2d77aaeb8f450592d678052d4a0038" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 41139 } }
I20260619 20:36:02.928131 26067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a2d77aaeb8f450592d678052d4a0038 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a2d77aaeb8f450592d678052d4a0038. Latest consensus state: current_term: 1 leader_uuid: "0a2d77aaeb8f450592d678052d4a0038" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a2d77aaeb8f450592d678052d4a0038" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 41139 } } }
I20260619 20:36:02.928207 26067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a2d77aaeb8f450592d678052d4a0038 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:02.928185 26069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a2d77aaeb8f450592d678052d4a0038 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a2d77aaeb8f450592d678052d4a0038" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a2d77aaeb8f450592d678052d4a0038" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 41139 } } }
I20260619 20:36:02.928290 26069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a2d77aaeb8f450592d678052d4a0038 [sys.catalog]: This master's current role is: LEADER
Jun 19 20:36:02 dist-test-slave-4m58 krb5kdc[25981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:36:02.928488 26074 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:36:02.928925 26074 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:36:02.931413 26074 catalog_manager.cc:1347] Generated new cluster ID: fac409bb2d8c44a5a4bdc45d7e1db2fb
I20260619 20:36:02.931478 26074 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:36:02.933053 26065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:02.921341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48429 (local address 127.4.83.126:41139)
0619 20:36:02.921557 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:02.921568 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:02.921603 (+ 35us) server_negotiation.cc:407] Connection header received
0619 20:36:02.922188 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:02.922199 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:02.922266 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:02.922399 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:02.923234 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:02.924627 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:02.925422 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:02.926060 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:02.926177 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:02.929191 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:02.929196 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:02.929197 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:02.929210 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:02.931224 (+ 2014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:02.931754 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:02.931757 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:02.931758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:02.931929 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:02.932153 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:02.932153 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:02.932154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:02.932431 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:02.932588 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:02.932728 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 20:36:02.932835 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":82,"threads_started":1}
I20260619 20:36:02.934106 4429 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:36:02.941088 26065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:02.935481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34809 (local address 127.4.83.126:41139)
0619 20:36:02.935516 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:02.935518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:02.935526 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:36:02.935576 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:02.935577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:02.935586 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:02.935626 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:02.936210 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:02.936704 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:02.937361 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:02.937598 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:02.937652 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:02.938777 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:02.938778 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:02.938779 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:02.938786 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:02.940036 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:02.940373 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:02.940374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:02.940375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:02.940423 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:02.940607 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:02.940608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:02.940608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:02.940715 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:02.940807 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:02.940937 (+ 130us) server_negotiation.cc:299] Negotiation successful
0619 20:36:02.940997 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:36:02.948560 26074 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:36:02.949306 26074 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:36:02.963912 26074 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a2d77aaeb8f450592d678052d4a0038: Generated new TSK 0
I20260619 20:36:02.964149 26074 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:36:02.966774 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 25994
W20260619 20:36:02.970515 25979 connection.cc:535] client connection to 127.4.83.126:41139 recv error: Network error: failed to read from TLS socket (remote: 127.4.83.126:41139): Connection reset by peer (error 104)
2026-06-19T20:36:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1251 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:36:03 dist-test-slave-4m58 krb5kdc[26093](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:36:03 dist-test-slave-4m58 krb5kdc[26093](info): set up 2 sockets
Jun 19 20:36:03 dist-test-slave-4m58 krb5kdc[26093](info): commencing operation
krb5kdc: starting...
W20260619 20:36:03.940927 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.957s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:36:04 dist-test-slave-4m58 krb5kdc[26093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:36:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:36:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
I20260619 20:36:04.075354 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:44201
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:34383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:44201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:04.154610 26106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:04.154834 26106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:04.154865 26106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:04.156050 26106 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:04.156098 26106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:04.156117 26106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:04.156132 26106 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:04.156149 26106 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:04.157673 26106 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:34383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:44201
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:44201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:04.157927 26106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:04.158136 26106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:04.163146 26112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:04.163146 26111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:04.163338 26106 server_base.cc:1034] running on GCE node
W20260619 20:36:04.163213 26114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:04.163702 26106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:04.164125 26106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:04.165347 26106 hybrid_clock.cc:648] HybridClock initialized: now 1781901364165316 us; error 50 us; skew 500 ppm
Jun 19 20:36:04 dist-test-slave-4m58 krb5kdc[26093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901364, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:04.171327 26106 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:36:04.171815 26106 webserver.cc:460] Webserver started at http://127.4.83.126:43125/ using document root <none> and password file <none>
I20260619 20:36:04.172081 26106 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:04.172135 26106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:04.172266 26106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:04.173333 26106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "6b3ffb5219bf475a8fe1bcab03ff670e"
format_stamp: "Formatted at 2026-06-19 20:36:04 on dist-test-slave-4m58"
server_key: "0d3d584ca69d90a3795c155f988a1b46"
server_key_iv: "29addf6cf7c3ea6fc6ff88a4ca2231c0"
server_key_version: "encryptionkey@0"
I20260619 20:36:04.173650 26106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "6b3ffb5219bf475a8fe1bcab03ff670e"
format_stamp: "Formatted at 2026-06-19 20:36:04 on dist-test-slave-4m58"
server_key: "0d3d584ca69d90a3795c155f988a1b46"
server_key_iv: "29addf6cf7c3ea6fc6ff88a4ca2231c0"
server_key_version: "encryptionkey@0"
I20260619 20:36:04.174945 26106 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:36:04.175737 26121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:04.175956 26106 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:36:04.176054 26106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "6b3ffb5219bf475a8fe1bcab03ff670e"
format_stamp: "Formatted at 2026-06-19 20:36:04 on dist-test-slave-4m58"
server_key: "0d3d584ca69d90a3795c155f988a1b46"
server_key_iv: "29addf6cf7c3ea6fc6ff88a4ca2231c0"
server_key_version: "encryptionkey@0"
I20260619 20:36:04.176111 26106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:04.195878 26106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:04.198055 26106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:04.198195 26106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:04.201936 26106 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:44201
I20260619 20:36:04.201984 26173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:44201 every 8 connection(s)
I20260619 20:36:04.202304 26106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:36:04.203142 26174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:04.205372 26174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b3ffb5219bf475a8fe1bcab03ff670e: Bootstrap starting.
I20260619 20:36:04.205915 26174 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b3ffb5219bf475a8fe1bcab03ff670e: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:04.206169 26174 log.cc:824] T 00000000000000000000000000000000 P 6b3ffb5219bf475a8fe1bcab03ff670e: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:04.206858 26174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b3ffb5219bf475a8fe1bcab03ff670e: No bootstrap required, opened a new log
I20260619 20:36:04.208442 26174 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b3ffb5219bf475a8fe1bcab03ff670e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b3ffb5219bf475a8fe1bcab03ff670e" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 44201 } }
I20260619 20:36:04.208582 26174 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b3ffb5219bf475a8fe1bcab03ff670e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:04.208613 26174 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b3ffb5219bf475a8fe1bcab03ff670e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b3ffb5219bf475a8fe1bcab03ff670e, State: Initialized, Role: FOLLOWER
I20260619 20:36:04.208746 26174 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b3ffb5219bf475a8fe1bcab03ff670e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b3ffb5219bf475a8fe1bcab03ff670e" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 44201 } }
I20260619 20:36:04.208807 26174 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b3ffb5219bf475a8fe1bcab03ff670e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:36:04.208828 26174 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b3ffb5219bf475a8fe1bcab03ff670e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:36:04.208874 26174 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b3ffb5219bf475a8fe1bcab03ff670e [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:04.209673 26174 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b3ffb5219bf475a8fe1bcab03ff670e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b3ffb5219bf475a8fe1bcab03ff670e" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 44201 } }
I20260619 20:36:04.209762 26174 leader_election.cc:302] T 00000000000000000000000000000000 P 6b3ffb5219bf475a8fe1bcab03ff670e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b3ffb5219bf475a8fe1bcab03ff670e; no voters:
I20260619 20:36:04.209923 26174 leader_election.cc:288] T 00000000000000000000000000000000 P 6b3ffb5219bf475a8fe1bcab03ff670e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:36:04.209965 26177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b3ffb5219bf475a8fe1bcab03ff670e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:04.210114 26177 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b3ffb5219bf475a8fe1bcab03ff670e [term 1 LEADER]: Becoming Leader. State: Replica: 6b3ffb5219bf475a8fe1bcab03ff670e, State: Running, Role: LEADER
I20260619 20:36:04.210186 26174 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b3ffb5219bf475a8fe1bcab03ff670e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:04.210206 26177 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b3ffb5219bf475a8fe1bcab03ff670e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b3ffb5219bf475a8fe1bcab03ff670e" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 44201 } }
I20260619 20:36:04.210551 26179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b3ffb5219bf475a8fe1bcab03ff670e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b3ffb5219bf475a8fe1bcab03ff670e. Latest consensus state: current_term: 1 leader_uuid: "6b3ffb5219bf475a8fe1bcab03ff670e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b3ffb5219bf475a8fe1bcab03ff670e" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 44201 } } }
I20260619 20:36:04.210641 26179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b3ffb5219bf475a8fe1bcab03ff670e [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:04.210546 26178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b3ffb5219bf475a8fe1bcab03ff670e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b3ffb5219bf475a8fe1bcab03ff670e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b3ffb5219bf475a8fe1bcab03ff670e" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 44201 } } }
I20260619 20:36:04.210817 26178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b3ffb5219bf475a8fe1bcab03ff670e [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:04.210881 26184 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:36:04.211374 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 26106
I20260619 20:36:04.211413 26184 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:36:04.211494 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:36:04.211710 4429 external_mini_cluster.cc:1428] Setting key 271772668cb7ba8953763f75b2a0316c
I20260619 20:36:04.214180 26184 catalog_manager.cc:1347] Generated new cluster ID: 3fda336a3c834fc7837e5aef5fd49516
I20260619 20:36:04.214237 26184 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:36:04.217228 26196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:04.212172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38123 (local address 127.4.83.126:44201)
0619 20:36:04.212584 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:04.212590 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:04.212613 (+ 23us) server_negotiation.cc:407] Connection header received
0619 20:36:04.212707 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:04.212715 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:04.212775 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:04.212911 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:04.213142 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:04.214045 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:04.215103 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:04.215733 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:04.215871 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:04.216555 (+ 684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:04.216559 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:04.216559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:36:04.216762 (+ 203us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:04.216801 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:36:04.216816 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:04.216906 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:04.216973 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 20:36:04.217002 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":62,"threads_started":1}
I20260619 20:36:04.218171 4429 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:36:04.220815 26184 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:36:04.221292 26184 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:36:04.223316 26196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:04.219556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57077 (local address 127.4.83.126:44201)
0619 20:36:04.219593 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:04.219595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:04.219733 (+ 138us) server_negotiation.cc:407] Connection header received
0619 20:36:04.219824 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:04.219825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:04.219861 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:04.219890 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:04.220098 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:04.220837 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:04.221897 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:04.222213 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:04.222253 (+ 40us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:04.223031 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:04.223033 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:04.223033 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:36:04.223037 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:04.223053 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:36:04.223057 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:04.223100 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:04.223225 (+ 125us) server_negotiation.cc:299] Negotiation successful
0619 20:36:04.223239 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 20:36:04.225219 26184 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b3ffb5219bf475a8fe1bcab03ff670e: Generated new TSK 0
I20260619 20:36:04.225383 26184 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:36:04.228911 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 26106
W20260619 20:36:04.233366 26091 connection.cc:535] client connection to 127.4.83.126:44201 recv error: Network error: failed to read from TLS socket (remote: 127.4.83.126:44201): Connection reset by peer (error 104)
2026-06-19T20:36:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1262 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:36:04 dist-test-slave-4m58 krb5kdc[26205](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:36:04 dist-test-slave-4m58 krb5kdc[26205](info): set up 2 sockets
Jun 19 20:36:04 dist-test-slave-4m58 krb5kdc[26205](info): commencing operation
krb5kdc: starting...
W20260619 20:36:05.192862 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.946s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:36:05 dist-test-slave-4m58 krb5kdc[26205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:36:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:36:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
I20260619 20:36:05.311291 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:35497
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:39299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:35497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:05.395141 26218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:05.395385 26218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:05.395421 26218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:05.396701 26218 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:05.396752 26218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:05.396772 26218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:05.396788 26218 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:05.396804 26218 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:05.398399 26218 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:39299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:35497
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:35497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:05.398671 26218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:05.398918 26218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:05.403637 26224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:05.403653 26223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:05.403752 26218 server_base.cc:1034] running on GCE node
W20260619 20:36:05.403895 26226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:05.404129 26218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:05.404394 26218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:05.405579 26218 hybrid_clock.cc:648] HybridClock initialized: now 1781901365405565 us; error 32 us; skew 500 ppm
Jun 19 20:36:05 dist-test-slave-4m58 krb5kdc[26205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901365, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:05.408363 26218 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:36:05.408847 26218 webserver.cc:460] Webserver started at http://127.4.83.126:46303/ using document root <none> and password file <none>
I20260619 20:36:05.409090 26218 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:05.409137 26218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:05.409271 26218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:05.410326 26218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "85ca0a8c1e3a4091bbb15074995e5102"
format_stamp: "Formatted at 2026-06-19 20:36:05 on dist-test-slave-4m58"
server_key: "a21cc901e85b36676af0c5692637e738"
server_key_iv: "9b816af659139e0301c16a1068365a04"
server_key_version: "encryptionkey@0"
I20260619 20:36:05.410656 26218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "85ca0a8c1e3a4091bbb15074995e5102"
format_stamp: "Formatted at 2026-06-19 20:36:05 on dist-test-slave-4m58"
server_key: "a21cc901e85b36676af0c5692637e738"
server_key_iv: "9b816af659139e0301c16a1068365a04"
server_key_version: "encryptionkey@0"
I20260619 20:36:05.412180 26218 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 20:36:05.412881 26233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:05.413035 26218 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:36:05.413121 26218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "85ca0a8c1e3a4091bbb15074995e5102"
format_stamp: "Formatted at 2026-06-19 20:36:05 on dist-test-slave-4m58"
server_key: "a21cc901e85b36676af0c5692637e738"
server_key_iv: "9b816af659139e0301c16a1068365a04"
server_key_version: "encryptionkey@0"
I20260619 20:36:05.413184 26218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:05.416493 26218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:05.418574 26218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:05.418686 26218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:05.422498 26218 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:35497
I20260619 20:36:05.422530 26285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:35497 every 8 connection(s)
I20260619 20:36:05.423090 26218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:36:05.424296 26286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:05.426271 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 26218
I20260619 20:36:05.426383 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:36:05.426577 4429 external_mini_cluster.cc:1428] Setting key 8836e32bc2711c4d40daef430c1dcd12
I20260619 20:36:05.426887 26286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85ca0a8c1e3a4091bbb15074995e5102: Bootstrap starting.
I20260619 20:36:05.427608 26286 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85ca0a8c1e3a4091bbb15074995e5102: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:05.428000 26286 log.cc:824] T 00000000000000000000000000000000 P 85ca0a8c1e3a4091bbb15074995e5102: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:05.428826 26286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85ca0a8c1e3a4091bbb15074995e5102: No bootstrap required, opened a new log
I20260619 20:36:05.429430 26289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:05.427191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47743 (local address 127.4.83.126:35497)
0619 20:36:05.427403 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:05.427412 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:05.427445 (+ 33us) server_negotiation.cc:407] Connection header received
0619 20:36:05.428055 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:05.428064 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:05.428109 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:05.428260 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:05.429015 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:05.429019 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:05.429019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:36:05.429024 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:05.429082 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:36:05.429093 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:05.429194 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:05.429227 (+ 33us) server_negotiation.cc:299] Negotiation successful
0619 20:36:05.429266 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":90,"threads_started":1}
I20260619 20:36:05.430289 4429 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:36:05.431054 26286 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85ca0a8c1e3a4091bbb15074995e5102 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85ca0a8c1e3a4091bbb15074995e5102" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35497 } }
I20260619 20:36:05.431223 26286 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85ca0a8c1e3a4091bbb15074995e5102 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:05.431267 26286 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85ca0a8c1e3a4091bbb15074995e5102 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85ca0a8c1e3a4091bbb15074995e5102, State: Initialized, Role: FOLLOWER
I20260619 20:36:05.431396 26286 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85ca0a8c1e3a4091bbb15074995e5102 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85ca0a8c1e3a4091bbb15074995e5102" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35497 } }
I20260619 20:36:05.431488 26286 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85ca0a8c1e3a4091bbb15074995e5102 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:36:05.431545 26286 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85ca0a8c1e3a4091bbb15074995e5102 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:36:05.431591 26286 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85ca0a8c1e3a4091bbb15074995e5102 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:05.432469 26286 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85ca0a8c1e3a4091bbb15074995e5102 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85ca0a8c1e3a4091bbb15074995e5102" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35497 } }
I20260619 20:36:05.432591 26286 leader_election.cc:302] T 00000000000000000000000000000000 P 85ca0a8c1e3a4091bbb15074995e5102 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85ca0a8c1e3a4091bbb15074995e5102; no voters:
I20260619 20:36:05.432766 26286 leader_election.cc:288] T 00000000000000000000000000000000 P 85ca0a8c1e3a4091bbb15074995e5102 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:36:05.432889 26289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:05.431606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47693 (local address 127.4.83.126:35497)
0619 20:36:05.431644 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:05.431645 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:05.431791 (+ 146us) server_negotiation.cc:407] Connection header received
0619 20:36:05.431817 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:05.431818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:05.431823 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:05.431994 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:05.432611 (+ 617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:05.432612 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:05.432612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:36:05.432616 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:05.432632 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:36:05.432636 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:05.432692 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:05.432789 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 20:36:05.432804 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:36:05.433043 26286 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85ca0a8c1e3a4091bbb15074995e5102 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:05.432797 26296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85ca0a8c1e3a4091bbb15074995e5102 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:05.433210 26296 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85ca0a8c1e3a4091bbb15074995e5102 [term 1 LEADER]: Becoming Leader. State: Replica: 85ca0a8c1e3a4091bbb15074995e5102, State: Running, Role: LEADER
I20260619 20:36:05.433459 26296 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85ca0a8c1e3a4091bbb15074995e5102 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85ca0a8c1e3a4091bbb15074995e5102" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35497 } }
I20260619 20:36:05.433859 26298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85ca0a8c1e3a4091bbb15074995e5102 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85ca0a8c1e3a4091bbb15074995e5102. Latest consensus state: current_term: 1 leader_uuid: "85ca0a8c1e3a4091bbb15074995e5102" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85ca0a8c1e3a4091bbb15074995e5102" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35497 } } }
I20260619 20:36:05.433944 26298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85ca0a8c1e3a4091bbb15074995e5102 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:05.434038 26297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85ca0a8c1e3a4091bbb15074995e5102 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85ca0a8c1e3a4091bbb15074995e5102" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85ca0a8c1e3a4091bbb15074995e5102" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 35497 } } }
I20260619 20:36:05.434094 26297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85ca0a8c1e3a4091bbb15074995e5102 [sys.catalog]: This master's current role is: LEADER
W20260619 20:36:05.434718 26311 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 85ca0a8c1e3a4091bbb15074995e5102: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:36:05.434758 26311 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:36:05.434827 26306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:36:05.435012 26306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:36:05.436631 26306 catalog_manager.cc:1347] Generated new cluster ID: ef6fa03f72a54894a9e200c78099a66d
I20260619 20:36:05.436684 26306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:36:05.444447 26306 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:36:05.444625 26306 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:36:05.446879 26306 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85ca0a8c1e3a4091bbb15074995e5102: Generated new TSK 0
I20260619 20:36:05.447043 26306 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:36:05.451364 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 26218
2026-06-19T20:36:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1221 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:36:05 dist-test-slave-4m58 krb5kdc[26317](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:36:05 dist-test-slave-4m58 krb5kdc[26317](info): set up 2 sockets
Jun 19 20:36:05 dist-test-slave-4m58 krb5kdc[26317](info): commencing operation
krb5kdc: starting...
W20260619 20:36:06.416709 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.948s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:36:06 dist-test-slave-4m58 krb5kdc[26317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:36:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:36:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
I20260619 20:36:06.547600 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:37873
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:45841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:37873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:06.627461 26330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:06.627697 26330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:06.627720 26330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:06.628902 26330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:06.628957 26330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:06.628970 26330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:06.628981 26330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:06.628993 26330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:06.630481 26330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:45841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:37873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:37873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:06.630697 26330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:06.630877 26330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:06.636441 26338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:06.636474 26330 server_base.cc:1034] running on GCE node
W20260619 20:36:06.636487 26336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:06.636441 26335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:06.637255 26330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:06.637635 26330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:06.638823 26330 hybrid_clock.cc:648] HybridClock initialized: now 1781901366638778 us; error 61 us; skew 500 ppm
Jun 19 20:36:06 dist-test-slave-4m58 krb5kdc[26317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901366, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:06.642064 26330 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:36:06.642854 26330 webserver.cc:460] Webserver started at http://127.4.83.126:46565/ using document root <none> and password file <none>
I20260619 20:36:06.643311 26330 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:06.643379 26330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:06.643545 26330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:06.645031 26330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "4bb0a4682b4d474cac59ddf4bbf5d0d2"
format_stamp: "Formatted at 2026-06-19 20:36:06 on dist-test-slave-4m58"
server_key: "de89dd0f00915184d017834941f282fd"
server_key_iv: "a8b2aa0b3de18a2babfe506707da8270"
server_key_version: "encryptionkey@0"
I20260619 20:36:06.645490 26330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "4bb0a4682b4d474cac59ddf4bbf5d0d2"
format_stamp: "Formatted at 2026-06-19 20:36:06 on dist-test-slave-4m58"
server_key: "de89dd0f00915184d017834941f282fd"
server_key_iv: "a8b2aa0b3de18a2babfe506707da8270"
server_key_version: "encryptionkey@0"
I20260619 20:36:06.647506 26330 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260619 20:36:06.648533 26345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:06.648751 26330 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:06.648854 26330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "4bb0a4682b4d474cac59ddf4bbf5d0d2"
format_stamp: "Formatted at 2026-06-19 20:36:06 on dist-test-slave-4m58"
server_key: "de89dd0f00915184d017834941f282fd"
server_key_iv: "a8b2aa0b3de18a2babfe506707da8270"
server_key_version: "encryptionkey@0"
I20260619 20:36:06.648942 26330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:06.658955 26330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:06.661072 26330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:06.661187 26330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:06.665457 26330 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:37873
I20260619 20:36:06.665555 26397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:37873 every 8 connection(s)
I20260619 20:36:06.665894 26330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:36:06.666987 26398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:06.670140 26398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bb0a4682b4d474cac59ddf4bbf5d0d2: Bootstrap starting.
I20260619 20:36:06.670882 26398 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4bb0a4682b4d474cac59ddf4bbf5d0d2: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:06.671201 26398 log.cc:824] T 00000000000000000000000000000000 P 4bb0a4682b4d474cac59ddf4bbf5d0d2: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:06.672231 26398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bb0a4682b4d474cac59ddf4bbf5d0d2: No bootstrap required, opened a new log
I20260619 20:36:06.672607 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 26330
I20260619 20:36:06.672726 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:36:06.672912 4429 external_mini_cluster.cc:1428] Setting key f4a3f7252abb7baefa3da9636bd8a8d7
I20260619 20:36:06.674695 26398 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4bb0a4682b4d474cac59ddf4bbf5d0d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bb0a4682b4d474cac59ddf4bbf5d0d2" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37873 } }
I20260619 20:36:06.674944 26398 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4bb0a4682b4d474cac59ddf4bbf5d0d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:06.674985 26398 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4bb0a4682b4d474cac59ddf4bbf5d0d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bb0a4682b4d474cac59ddf4bbf5d0d2, State: Initialized, Role: FOLLOWER
I20260619 20:36:06.675194 26398 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4bb0a4682b4d474cac59ddf4bbf5d0d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bb0a4682b4d474cac59ddf4bbf5d0d2" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37873 } }
I20260619 20:36:06.675271 26398 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4bb0a4682b4d474cac59ddf4bbf5d0d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:36:06.675297 26398 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4bb0a4682b4d474cac59ddf4bbf5d0d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:36:06.675331 26398 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4bb0a4682b4d474cac59ddf4bbf5d0d2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:06.676775 26398 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4bb0a4682b4d474cac59ddf4bbf5d0d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bb0a4682b4d474cac59ddf4bbf5d0d2" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37873 } }
I20260619 20:36:06.676939 26398 leader_election.cc:302] T 00000000000000000000000000000000 P 4bb0a4682b4d474cac59ddf4bbf5d0d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4bb0a4682b4d474cac59ddf4bbf5d0d2; no voters:
I20260619 20:36:06.677147 26398 leader_election.cc:288] T 00000000000000000000000000000000 P 4bb0a4682b4d474cac59ddf4bbf5d0d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:36:06.677234 26403 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4bb0a4682b4d474cac59ddf4bbf5d0d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:06.677489 26403 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4bb0a4682b4d474cac59ddf4bbf5d0d2 [term 1 LEADER]: Becoming Leader. State: Replica: 4bb0a4682b4d474cac59ddf4bbf5d0d2, State: Running, Role: LEADER
I20260619 20:36:06.677528 26398 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4bb0a4682b4d474cac59ddf4bbf5d0d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:06.677728 26403 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4bb0a4682b4d474cac59ddf4bbf5d0d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bb0a4682b4d474cac59ddf4bbf5d0d2" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37873 } }
I20260619 20:36:06.678257 26404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bb0a4682b4d474cac59ddf4bbf5d0d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4bb0a4682b4d474cac59ddf4bbf5d0d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bb0a4682b4d474cac59ddf4bbf5d0d2" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37873 } } }
I20260619 20:36:06.678350 26404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bb0a4682b4d474cac59ddf4bbf5d0d2 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:06.678603 26405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bb0a4682b4d474cac59ddf4bbf5d0d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4bb0a4682b4d474cac59ddf4bbf5d0d2. Latest consensus state: current_term: 1 leader_uuid: "4bb0a4682b4d474cac59ddf4bbf5d0d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bb0a4682b4d474cac59ddf4bbf5d0d2" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37873 } } }
I20260619 20:36:06.678720 26405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bb0a4682b4d474cac59ddf4bbf5d0d2 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:06.679085 26412 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:36:06.679734 26412 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 20:36:06 dist-test-slave-4m58 krb5kdc[26317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:36:06.682037 26412 catalog_manager.cc:1347] Generated new cluster ID: c17df1a729ae414597a29760787aeca4
I20260619 20:36:06.682101 26412 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:36:06.686019 26402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:06.673524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41343 (local address 127.4.83.126:37873)
0619 20:36:06.673799 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:06.673811 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:06.673857 (+ 46us) server_negotiation.cc:407] Connection header received
0619 20:36:06.674637 (+ 780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:06.674648 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:06.674728 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:06.674934 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:06.675710 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:06.677210 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:06.678322 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:06.678971 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:06.679071 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:06.681999 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:06.682003 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:06.682003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:06.682016 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:06.684072 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:06.684649 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:06.684652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:06.684652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:06.684788 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:06.685028 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:06.685029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:06.685029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:06.685335 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:06.685494 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:06.685659 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 20:36:06.685812 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":138,"threads_started":1}
I20260619 20:36:06.687170 4429 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:36:06.692548 26412 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:36:06.693459 26412 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:36:06.702924 26402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:06.692375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35361 (local address 127.4.83.126:37873)
0619 20:36:06.692501 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:06.692503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:06.696122 (+ 3619us) server_negotiation.cc:407] Connection header received
0619 20:36:06.696194 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:06.696196 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:06.696210 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:06.696260 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:06.696865 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:06.697417 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:06.698122 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:06.698356 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:06.698411 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:06.699788 (+ 1377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:06.699789 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:06.699790 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:06.699799 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:06.701734 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:06.702058 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:06.702059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:06.702059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:06.702119 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:06.702301 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:06.702302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:06.702302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:06.702449 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:06.702544 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:06.702741 (+ 197us) server_negotiation.cc:299] Negotiation successful
0619 20:36:06.702825 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260619 20:36:06.707976 26412 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4bb0a4682b4d474cac59ddf4bbf5d0d2: Generated new TSK 0
I20260619 20:36:06.708230 26412 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:36:06.710868 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 26330
2026-06-19T20:36:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1260 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:36:06 dist-test-slave-4m58 krb5kdc[26429](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:36:06 dist-test-slave-4m58 krb5kdc[26429](info): set up 2 sockets
Jun 19 20:36:06 dist-test-slave-4m58 krb5kdc[26429](info): commencing operation
krb5kdc: starting...
W20260619 20:36:07.676683 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.947s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:36:07 dist-test-slave-4m58 krb5kdc[26429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:36:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:36:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
I20260619 20:36:07.790942 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:42261
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:42093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:42261
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:07.883765 26442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:07.884037 26442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:07.884068 26442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:07.885418 26442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:07.885471 26442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:07.885491 26442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:07.885507 26442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:07.885524 26442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:07.887300 26442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:42093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:42261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:42261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:07.887584 26442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:07.887804 26442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:07.892827 26448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:07.892839 26450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:07.892899 26442 server_base.cc:1034] running on GCE node
W20260619 20:36:07.892848 26447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:07.893293 26442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:07.893570 26442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:07.894722 26442 hybrid_clock.cc:648] HybridClock initialized: now 1781901367894706 us; error 38 us; skew 500 ppm
Jun 19 20:36:07 dist-test-slave-4m58 krb5kdc[26429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901367, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:07.897100 26442 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:36:07.897552 26442 webserver.cc:460] Webserver started at http://127.4.83.126:34667/ using document root <none> and password file <none>
I20260619 20:36:07.897766 26442 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:07.897817 26442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:07.897943 26442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:07.898978 26442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "6ef4ec04610a48ecae6099f1d376021d"
format_stamp: "Formatted at 2026-06-19 20:36:07 on dist-test-slave-4m58"
server_key: "bcc33a2a004095159e1115d0b00291c5"
server_key_iv: "7e86b47f84ab66733973ade259a82275"
server_key_version: "encryptionkey@0"
I20260619 20:36:07.899312 26442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "6ef4ec04610a48ecae6099f1d376021d"
format_stamp: "Formatted at 2026-06-19 20:36:07 on dist-test-slave-4m58"
server_key: "bcc33a2a004095159e1115d0b00291c5"
server_key_iv: "7e86b47f84ab66733973ade259a82275"
server_key_version: "encryptionkey@0"
I20260619 20:36:07.900657 26442 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:36:07.901362 26457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:07.901549 26442 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:07.901652 26442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "6ef4ec04610a48ecae6099f1d376021d"
format_stamp: "Formatted at 2026-06-19 20:36:07 on dist-test-slave-4m58"
server_key: "bcc33a2a004095159e1115d0b00291c5"
server_key_iv: "7e86b47f84ab66733973ade259a82275"
server_key_version: "encryptionkey@0"
I20260619 20:36:07.901710 26442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:07.916309 26442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:07.918375 26442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:07.918517 26442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:07.922194 26442 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:42261
I20260619 20:36:07.922307 26509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:42261 every 8 connection(s)
I20260619 20:36:07.922549 26442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:36:07.923365 26510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:07.925473 26510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ef4ec04610a48ecae6099f1d376021d: Bootstrap starting.
I20260619 20:36:07.925850 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 26442
I20260619 20:36:07.925956 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:36:07.926123 26510 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ef4ec04610a48ecae6099f1d376021d: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:07.926136 4429 external_mini_cluster.cc:1428] Setting key 96e910002a6abf3fb43b3ffa9a28bbef
I20260619 20:36:07.926409 26510 log.cc:824] T 00000000000000000000000000000000 P 6ef4ec04610a48ecae6099f1d376021d: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:07.927141 26510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ef4ec04610a48ecae6099f1d376021d: No bootstrap required, opened a new log
I20260619 20:36:07.929558 26510 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ef4ec04610a48ecae6099f1d376021d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ef4ec04610a48ecae6099f1d376021d" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 42261 } }
I20260619 20:36:07.929697 26510 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ef4ec04610a48ecae6099f1d376021d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:07.929718 26510 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ef4ec04610a48ecae6099f1d376021d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ef4ec04610a48ecae6099f1d376021d, State: Initialized, Role: FOLLOWER
I20260619 20:36:07.929824 26510 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ef4ec04610a48ecae6099f1d376021d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ef4ec04610a48ecae6099f1d376021d" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 42261 } }
I20260619 20:36:07.929894 26510 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ef4ec04610a48ecae6099f1d376021d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:36:07.929920 26510 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ef4ec04610a48ecae6099f1d376021d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:36:07.929965 26510 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ef4ec04610a48ecae6099f1d376021d [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:07.930645 26510 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ef4ec04610a48ecae6099f1d376021d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ef4ec04610a48ecae6099f1d376021d" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 42261 } }
I20260619 20:36:07.930751 26510 leader_election.cc:302] T 00000000000000000000000000000000 P 6ef4ec04610a48ecae6099f1d376021d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ef4ec04610a48ecae6099f1d376021d; no voters:
I20260619 20:36:07.930923 26510 leader_election.cc:288] T 00000000000000000000000000000000 P 6ef4ec04610a48ecae6099f1d376021d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:36:07.931026 26515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ef4ec04610a48ecae6099f1d376021d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:07.931224 26510 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ef4ec04610a48ecae6099f1d376021d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:07.931332 26515 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ef4ec04610a48ecae6099f1d376021d [term 1 LEADER]: Becoming Leader. State: Replica: 6ef4ec04610a48ecae6099f1d376021d, State: Running, Role: LEADER
I20260619 20:36:07.931474 26515 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ef4ec04610a48ecae6099f1d376021d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ef4ec04610a48ecae6099f1d376021d" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 42261 } }
I20260619 20:36:07.931936 26516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ef4ec04610a48ecae6099f1d376021d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ef4ec04610a48ecae6099f1d376021d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ef4ec04610a48ecae6099f1d376021d" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 42261 } } }
I20260619 20:36:07.932042 26516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ef4ec04610a48ecae6099f1d376021d [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:07.932231 26517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ef4ec04610a48ecae6099f1d376021d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ef4ec04610a48ecae6099f1d376021d. Latest consensus state: current_term: 1 leader_uuid: "6ef4ec04610a48ecae6099f1d376021d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ef4ec04610a48ecae6099f1d376021d" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 42261 } } }
I20260619 20:36:07.932430 26517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ef4ec04610a48ecae6099f1d376021d [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:07.932582 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:07.926643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58485 (local address 127.4.83.126:42261)
0619 20:36:07.926817 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:07.926827 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:07.926859 (+ 32us) server_negotiation.cc:407] Connection header received
0619 20:36:07.927437 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:07.927446 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:07.927507 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:07.927663 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:07.927869 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:07.929292 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:07.930254 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:07.930879 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:07.930991 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:07.931736 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:07.931740 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:07.931741 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:36:07.931977 (+ 236us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:07.932017 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:36:07.932032 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:07.932094 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:07.932315 (+ 221us) server_negotiation.cc:299] Negotiation successful
0619 20:36:07.932383 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
I20260619 20:36:07.932569 26524 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:36:07.933157 26524 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:36:07.933656 4429 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:36:07.935593 26524 catalog_manager.cc:1347] Generated new cluster ID: 0928938fdeff4b1e8a43821f201bbd5e
I20260619 20:36:07.935643 26524 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:36:07.939218 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:07.934869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35739 (local address 127.4.83.126:42261)
0619 20:36:07.934928 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:07.934930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:07.936009 (+ 1079us) server_negotiation.cc:407] Connection header received
0619 20:36:07.936075 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:07.936076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:07.936087 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:07.936143 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:07.936280 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:07.936998 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:07.937995 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:07.938272 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:07.938325 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:07.939012 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:07.939013 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:07.939013 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:36:07.939017 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:07.939031 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:36:07.939035 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:07.939078 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:07.939122 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 20:36:07.939142 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 20:36:07.959470 26524 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:36:07.960284 26524 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:36:07.965905 26524 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ef4ec04610a48ecae6099f1d376021d: Generated new TSK 0
I20260619 20:36:07.966137 26524 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:36:07.972949 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 26442
2026-06-19T20:36:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1260 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:36:08 dist-test-slave-4m58 krb5kdc[26541](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:36:08 dist-test-slave-4m58 krb5kdc[26541](info): set up 2 sockets
Jun 19 20:36:08 dist-test-slave-4m58 krb5kdc[26541](info): commencing operation
krb5kdc: starting...
W20260619 20:36:08.940750 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.951s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:36:09 dist-test-slave-4m58 krb5kdc[26541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:36:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:36:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
I20260619 20:36:09.055308 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:34889
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:39713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:34889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:09.134495 26554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:09.134727 26554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:09.134749 26554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:09.135926 26554 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:09.135967 26554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:09.135980 26554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:09.135991 26554 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:09.136003 26554 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:09.137465 26554 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:39713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:34889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:34889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:09.137670 26554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:09.137857 26554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:09.142570 26559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:09.142549 26562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:09.142549 26560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:09.142886 26554 server_base.cc:1034] running on GCE node
I20260619 20:36:09.143146 26554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:09.143453 26554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:09.144659 26554 hybrid_clock.cc:648] HybridClock initialized: now 1781901369144625 us; error 47 us; skew 500 ppm
Jun 19 20:36:09 dist-test-slave-4m58 krb5kdc[26541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901369, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:09.147327 26554 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:36:09.147881 26554 webserver.cc:460] Webserver started at http://127.4.83.126:39531/ using document root <none> and password file <none>
I20260619 20:36:09.148099 26554 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:09.148149 26554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:09.148278 26554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:09.149358 26554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "e2a668c049434399a6e6699ceb5ea8d8"
format_stamp: "Formatted at 2026-06-19 20:36:09 on dist-test-slave-4m58"
server_key: "866168e1bf44db4fc383c9cb43abeb4f"
server_key_iv: "798b5b32a613d96d2bfc2a61b413e50f"
server_key_version: "encryptionkey@0"
I20260619 20:36:09.149719 26554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "e2a668c049434399a6e6699ceb5ea8d8"
format_stamp: "Formatted at 2026-06-19 20:36:09 on dist-test-slave-4m58"
server_key: "866168e1bf44db4fc383c9cb43abeb4f"
server_key_iv: "798b5b32a613d96d2bfc2a61b413e50f"
server_key_version: "encryptionkey@0"
I20260619 20:36:09.151101 26554 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:36:09.151948 26569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:09.152120 26554 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:09.152216 26554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "e2a668c049434399a6e6699ceb5ea8d8"
format_stamp: "Formatted at 2026-06-19 20:36:09 on dist-test-slave-4m58"
server_key: "866168e1bf44db4fc383c9cb43abeb4f"
server_key_iv: "798b5b32a613d96d2bfc2a61b413e50f"
server_key_version: "encryptionkey@0"
I20260619 20:36:09.152292 26554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:09.155643 26554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:09.158264 26554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:09.158420 26554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:09.162411 26554 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:34889
I20260619 20:36:09.162441 26621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:34889 every 8 connection(s)
I20260619 20:36:09.162763 26554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:36:09.163667 26622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:09.165905 26622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2a668c049434399a6e6699ceb5ea8d8: Bootstrap starting.
I20260619 20:36:09.166474 26622 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2a668c049434399a6e6699ceb5ea8d8: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:09.166725 26622 log.cc:824] T 00000000000000000000000000000000 P e2a668c049434399a6e6699ceb5ea8d8: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:09.167443 26622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2a668c049434399a6e6699ceb5ea8d8: No bootstrap required, opened a new log
I20260619 20:36:09.169111 26622 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2a668c049434399a6e6699ceb5ea8d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2a668c049434399a6e6699ceb5ea8d8" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 34889 } }
I20260619 20:36:09.169296 26622 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2a668c049434399a6e6699ceb5ea8d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:09.169320 26622 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2a668c049434399a6e6699ceb5ea8d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a668c049434399a6e6699ceb5ea8d8, State: Initialized, Role: FOLLOWER
I20260619 20:36:09.169459 26622 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2a668c049434399a6e6699ceb5ea8d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2a668c049434399a6e6699ceb5ea8d8" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 34889 } }
I20260619 20:36:09.169528 26622 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2a668c049434399a6e6699ceb5ea8d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:36:09.169548 26622 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2a668c049434399a6e6699ceb5ea8d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:36:09.169579 26622 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2a668c049434399a6e6699ceb5ea8d8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:09.170110 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 26554
I20260619 20:36:09.170222 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:36:09.170387 4429 external_mini_cluster.cc:1428] Setting key ac4b42cb956ef165e9a9e3e16981c165
I20260619 20:36:09.170720 26622 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2a668c049434399a6e6699ceb5ea8d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2a668c049434399a6e6699ceb5ea8d8" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 34889 } }
I20260619 20:36:09.171051 26622 leader_election.cc:302] T 00000000000000000000000000000000 P e2a668c049434399a6e6699ceb5ea8d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2a668c049434399a6e6699ceb5ea8d8; no voters:
I20260619 20:36:09.171229 26622 leader_election.cc:288] T 00000000000000000000000000000000 P e2a668c049434399a6e6699ceb5ea8d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:36:09.171273 26627 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2a668c049434399a6e6699ceb5ea8d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:09.171418 26627 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2a668c049434399a6e6699ceb5ea8d8 [term 1 LEADER]: Becoming Leader. State: Replica: e2a668c049434399a6e6699ceb5ea8d8, State: Running, Role: LEADER
I20260619 20:36:09.171504 26622 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2a668c049434399a6e6699ceb5ea8d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:09.171572 26627 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2a668c049434399a6e6699ceb5ea8d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2a668c049434399a6e6699ceb5ea8d8" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 34889 } }
I20260619 20:36:09.171953 26628 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2a668c049434399a6e6699ceb5ea8d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2a668c049434399a6e6699ceb5ea8d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2a668c049434399a6e6699ceb5ea8d8" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 34889 } } }
I20260619 20:36:09.172035 26628 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2a668c049434399a6e6699ceb5ea8d8 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:09.172199 26629 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2a668c049434399a6e6699ceb5ea8d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2a668c049434399a6e6699ceb5ea8d8. Latest consensus state: current_term: 1 leader_uuid: "e2a668c049434399a6e6699ceb5ea8d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2a668c049434399a6e6699ceb5ea8d8" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 34889 } } }
I20260619 20:36:09.172315 26629 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2a668c049434399a6e6699ceb5ea8d8 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:09.172536 26633 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:36:09.173142 26633 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:36:09.174687 26633 catalog_manager.cc:1347] Generated new cluster ID: c297c3c0f18c4da986252695c168b3bd
I20260619 20:36:09.174732 26633 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:36:09.179490 26626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:09.170903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49169 (local address 127.4.83.126:34889)
0619 20:36:09.171076 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:09.171087 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:09.171122 (+ 35us) server_negotiation.cc:407] Connection header received
0619 20:36:09.171253 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:09.171260 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:09.171296 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:09.171726 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:09.178611 (+ 6885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:09.178617 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:09.178617 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:36:09.178625 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:09.178701 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:36:09.178839 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:09.178944 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:09.179014 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 20:36:09.179075 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":72,"threads_started":1}
I20260619 20:36:09.180547 4429 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:36:09.183418 26626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:09.182229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42625 (local address 127.4.83.126:34889)
0619 20:36:09.182279 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:09.182281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:09.182451 (+ 170us) server_negotiation.cc:407] Connection header received
0619 20:36:09.182507 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:09.182508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:09.182514 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:09.182585 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:09.183197 (+ 612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:09.183199 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:09.183199 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:36:09.183203 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:09.183221 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:36:09.183226 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:09.183283 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:09.183330 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 20:36:09.183343 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:36:09.197852 26633 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:36:09.198041 26633 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:36:09.202988 26633 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2a668c049434399a6e6699ceb5ea8d8: Generated new TSK 0
I20260619 20:36:09.203194 26633 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:36:09.214139 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 26554
2026-06-19T20:36:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1241 ms)
[----------] 18 tests from AuthTokenIssuingTest (22927 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:36:09 dist-test-slave-4m58 krb5kdc[26653](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:36:09 dist-test-slave-4m58 krb5kdc[26653](info): set up 2 sockets
Jun 19 20:36:09 dist-test-slave-4m58 krb5kdc[26653](info): commencing operation
krb5kdc: starting...
W20260619 20:36:10.181121 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.950s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:36:10 dist-test-slave-4m58 krb5kdc[26653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:36:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:36:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
I20260619 20:36:10.299454 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:37903
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:37759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:37903,127.4.83.125:35755,127.4.83.124:40825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:10.382108 26666 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 20:36:10.382376 26666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:10.382403 26666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:10.382434 26666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:10.383728 26666 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:10.383781 26666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:10.383800 26666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:10.383816 26666 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:10.383858 26666 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:10.385529 26666 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:37759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:37903,127.4.83.125:35755,127.4.83.124:40825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:37903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:10.385793 26666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:10.386009 26666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:36:10.390854 26666 server_base.cc:1034] running on GCE node
W20260619 20:36:10.390913 26671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:10.390946 26672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:10.391141 26674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:10.391405 26666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:10.391717 26666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:10.392868 26666 hybrid_clock.cc:648] HybridClock initialized: now 1781901370392864 us; error 60 us; skew 500 ppm
Jun 19 20:36:10 dist-test-slave-4m58 krb5kdc[26653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901370, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:10.395321 26666 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:36:10.395766 26666 webserver.cc:460] Webserver started at http://127.4.83.126:35395/ using document root <none> and password file <none>
I20260619 20:36:10.395999 26666 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:10.396049 26666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:10.396178 26666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:10.397236 26666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "b44931b928464e7cbb7ab46f88a72f7a"
format_stamp: "Formatted at 2026-06-19 20:36:10 on dist-test-slave-4m58"
server_key: "af5373b9558210a1895a18f7fb32ca56"
server_key_iv: "c4fbad2f79f3b2c22b5906851f8d455b"
server_key_version: "encryptionkey@0"
I20260619 20:36:10.397564 26666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "b44931b928464e7cbb7ab46f88a72f7a"
format_stamp: "Formatted at 2026-06-19 20:36:10 on dist-test-slave-4m58"
server_key: "af5373b9558210a1895a18f7fb32ca56"
server_key_iv: "c4fbad2f79f3b2c22b5906851f8d455b"
server_key_version: "encryptionkey@0"
I20260619 20:36:10.398828 26666 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:36:10.399540 26681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:10.399716 26666 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:36:10.399799 26666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "b44931b928464e7cbb7ab46f88a72f7a"
format_stamp: "Formatted at 2026-06-19 20:36:10 on dist-test-slave-4m58"
server_key: "af5373b9558210a1895a18f7fb32ca56"
server_key_iv: "c4fbad2f79f3b2c22b5906851f8d455b"
server_key_version: "encryptionkey@0"
I20260619 20:36:10.399902 26666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:10.417207 26666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:10.419354 26666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:10.419495 26666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:10.423323 26666 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:37903
I20260619 20:36:10.423395 26733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:37903 every 8 connection(s)
I20260619 20:36:10.423723 26666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:36:10.424594 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 26666
I20260619 20:36:10.424741 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:36:10.424774 26734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:10.424921 4429 external_mini_cluster.cc:1428] Setting key 857959937fa83a8ba37032ddd118e07c
I20260619 20:36:10.426316 26734 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37903 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.4.83.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.125@KRBTEST.COM" created.
Jun 19 20:36:10 dist-test-slave-4m58 krb5kdc[26653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901370, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:36:10.438866 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.427006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:37903 (local address 127.0.0.1:33280)
0619 20:36:10.427165 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:10.427207 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:10.427692 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:10.427983 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:10.427985 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:10.428263 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:10.428408 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:10.428412 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.429394 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.429395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:10.430075 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:10.430077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.430567 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.430568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:10.430630 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.431211 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:10.431220 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:10.435216 (+ 3996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:10.437283 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:10.437290 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:10.437292 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:10.437492 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:10.437737 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:10.437737 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:10.437738 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:10.437811 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:10.438295 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:10.438302 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:10.438369 (+ 67us) client_negotiation.cc:769] Sending connection context
0619 20:36:10.438465 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 20:36:10.438616 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
I20260619 20:36:10.438872 26738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.427029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33280 (local address 127.4.83.126:37903)
0619 20:36:10.427224 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.427228 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.427287 (+ 59us) server_negotiation.cc:407] Connection header received
0619 20:36:10.427870 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.427873 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.427894 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.427974 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:10.428499 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.429310 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.430156 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.430475 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.430574 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.435331 (+ 4757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:10.435342 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:10.435342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:10.435350 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:10.437195 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:10.437567 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:10.437570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:10.437570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:10.437677 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:10.437861 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:10.437862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:10.437863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:10.438189 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:10.438293 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:10.438520 (+ 227us) server_negotiation.cc:299] Negotiation successful
0619 20:36:10.438693 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":173,"thread_start_us":83,"threads_started":1}
I20260619 20:36:10.439688 26734 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.125" port: 35755 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:36:10.440068 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.439959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:35755 (local address 127.0.0.1:59164)
0619 20:36:10.440007 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:10.440047 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.125:35755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260619 20:36:10.440188 26683 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.83.125:35755: connect: Connection refused (error 111)
I20260619 20:36:10.440840 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.440693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:35755 (local address 127.0.0.1:59180)
0619 20:36:10.440792 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:10.440818 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.125:35755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260619 20:36:10.440938 26734 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.83.125:35755: Network error: Client connection negotiation failed: client connection to 127.4.83.125:35755: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.125.keytab.
Entry for principal kudu/127.4.83.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.125.keytab.
I20260619 20:36:10.451895 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.125:35755
--webserver_interface=127.4.83.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.125.keytab
--principal=kudu/127.4.83.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:37759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:37903,127.4.83.125:35755,127.4.83.124:40825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:36:10.490166 26734 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.83.125" port: 35755 } attempt: 1
I20260619 20:36:10.490743 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.490580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:35755 (local address 127.0.0.1:59194)
0619 20:36:10.490651 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:10.490686 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.125:35755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 20:36:10.491091 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.491009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:35755 (local address 127.0.0.1:59200)
0619 20:36:10.491046 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:10.491068 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.125:35755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260619 20:36:10.491184 26734 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.83.125:35755: Network error: Client connection negotiation failed: client connection to 127.4.83.125:35755: connect: Connection refused (error 111)
W20260619 20:36:10.531730 26741 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 20:36:10.531991 26741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:10.532006 26741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:10.532029 26741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:10.533190 26741 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:10.533231 26741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:10.533242 26741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:10.533253 26741 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:10.533264 26741 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:10.534766 26741 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:37759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:37903,127.4.83.125:35755,127.4.83.124:40825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.125.keytab
--principal=kudu/127.4.83.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.125:35755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.83.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:10.534973 26741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:10.535156 26741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:10.540026 26749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:10.540033 26747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:10.540123 26741 server_base.cc:1034] running on GCE node
W20260619 20:36:10.540300 26746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:10.540596 26741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:10.540953 26741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:10.542124 26741 hybrid_clock.cc:648] HybridClock initialized: now 1781901370542082 us; error 66 us; skew 500 ppm
Jun 19 20:36:10 dist-test-slave-4m58 krb5kdc[26653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901370, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:10.544682 26741 init.cc:377] Logged in from keytab as kudu/127.4.83.125@KRBTEST.COM (short username kudu)
I20260619 20:36:10.545150 26741 webserver.cc:460] Webserver started at http://127.4.83.125:34969/ using document root <none> and password file <none>
I20260619 20:36:10.545356 26741 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:10.545403 26741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:10.545528 26741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:10.546612 26741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-1/data/instance:
uuid: "c31cdc424a274057957528660c92d89d"
format_stamp: "Formatted at 2026-06-19 20:36:10 on dist-test-slave-4m58"
server_key: "cc5493579a55619e37b29a7aec301dfb"
server_key_iv: "5e40884bfdf0cc676a31db29cabab027"
server_key_version: "encryptionkey@0"
I20260619 20:36:10.546934 26741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-1/wal/instance:
uuid: "c31cdc424a274057957528660c92d89d"
format_stamp: "Formatted at 2026-06-19 20:36:10 on dist-test-slave-4m58"
server_key: "cc5493579a55619e37b29a7aec301dfb"
server_key_iv: "5e40884bfdf0cc676a31db29cabab027"
server_key_version: "encryptionkey@0"
I20260619 20:36:10.548166 26741 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 20:36:10.548820 26756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:10.549000 26741 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:10.549090 26741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-1/wal
uuid: "c31cdc424a274057957528660c92d89d"
format_stamp: "Formatted at 2026-06-19 20:36:10 on dist-test-slave-4m58"
server_key: "cc5493579a55619e37b29a7aec301dfb"
server_key_iv: "5e40884bfdf0cc676a31db29cabab027"
server_key_version: "encryptionkey@0"
I20260619 20:36:10.549154 26741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:10.559329 26734 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.83.125" port: 35755 } attempt: 2
I20260619 20:36:10.559809 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.559647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:35755 (local address 127.0.0.1:59216)
0619 20:36:10.559737 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:10.559777 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.125:35755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260619 20:36:10.560207 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.560115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:35755 (local address 127.0.0.1:59224)
0619 20:36:10.560158 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:10.560185 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.125:35755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 20:36:10.560297 26734 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.83.125:35755: Network error: Client connection negotiation failed: client connection to 127.4.83.125:35755: connect: Connection refused (error 111)
I20260619 20:36:10.560994 26741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:10.563066 26741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:10.563213 26741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:10.566970 26741 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.125:35755
I20260619 20:36:10.567006 26808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.125:35755 every 8 connection(s)
I20260619 20:36:10.567339 26741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-1/data/info.pb
I20260619 20:36:10.568599 26809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:10.570215 26809 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37903 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:36:10.576692 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 26741
I20260619 20:36:10.576829 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-1/wal/instance
I20260619 20:36:10.577026 4429 external_mini_cluster.cc:1428] Setting key e67eb97db07f4bb41d98b050c61a37d1
Jun 19 20:36:10 dist-test-slave-4m58 krb5kdc[26653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901370, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.125@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:36:10.580708 26738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.571113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33282 (local address 127.4.83.126:37903)
0619 20:36:10.571224 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.571226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.571307 (+ 81us) server_negotiation.cc:407] Connection header received
0619 20:36:10.571966 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.571968 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.571986 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.572040 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:10.572688 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.573316 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.574883 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.575105 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.575170 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.577668 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:10.577670 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:10.577670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:10.577678 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:10.579256 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:10.579711 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:10.579713 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:10.579713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:10.579771 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:10.580035 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:10.580036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:10.580036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:10.580159 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:10.580225 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:10.580518 (+ 293us) server_negotiation.cc:299] Negotiation successful
0619 20:36:10.580608 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260619 20:36:10.580927 26811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.570948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:37903 (local address 127.0.0.1:33282)
0619 20:36:10.571163 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:10.571217 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:10.571819 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:10.572118 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:10.572121 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:10.572387 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:10.572596 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:10.572600 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.573518 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.573520 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:10.574744 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:10.574750 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.575218 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.575219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:10.575346 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.575923 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:10.575933 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:10.577545 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:10.579367 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:10.579378 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:10.579381 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:10.579636 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:10.579854 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:10.579855 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:10.579855 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:10.579975 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:10.580227 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:10.580234 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:10.580390 (+ 156us) client_negotiation.cc:769] Sending connection context
0619 20:36:10.580476 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 20:36:10.580660 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":102,"threads_started":1}
I20260619 20:36:10.581432 26809 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.125" port: 35755 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.4.83.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.124@KRBTEST.COM" created.
Jun 19 20:36:10 dist-test-slave-4m58 krb5kdc[26653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901370, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.125@KRBTEST.COM for kudu/127.4.83.125@KRBTEST.COM
I20260619 20:36:10.591477 26811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.581678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:35755 (local address 127.0.0.1:59238)
0619 20:36:10.581739 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:10.581753 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:10.581810 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:10.582023 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:10.582024 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:10.582175 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:10.582248 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:10.582250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.583239 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.583240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:10.584108 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:10.584111 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.584757 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.584757 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:10.584845 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.585286 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:10.585290 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:10.587775 (+ 2485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:10.590332 (+ 2557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:10.590333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:10.590334 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:10.590536 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:10.590771 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:10.590772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:10.590773 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:10.590835 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:10.591175 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:10.591176 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:10.591224 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 20:36:10.591289 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 20:36:10.591387 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 20:36:10.591583 26813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.581737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59238 (local address 127.4.83.125:35755)
0619 20:36:10.581891 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.581896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.581915 (+ 19us) server_negotiation.cc:407] Connection header received
0619 20:36:10.581934 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.581936 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.581947 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.582006 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:10.582358 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.583146 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.584225 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.584642 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.584702 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.587883 (+ 3181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:10.588008 (+ 125us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:10.588009 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:10.588019 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:10.590235 (+ 2216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:10.590617 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:10.590621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:10.590622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:10.590689 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:10.590898 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:10.590898 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:10.590899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:10.591092 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:10.591169 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:10.591336 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 20:36:10.591484 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
I20260619 20:36:10.592226 26809 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.124" port: 40825 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:36:10.592563 26811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.592434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:40825 (local address 127.0.0.1:38214)
0619 20:36:10.592480 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:10.592534 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:40825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 20:36:10.592707 26758 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.83.124:40825: connect: Connection refused (error 111)
I20260619 20:36:10.593161 26811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.593069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:40825 (local address 127.0.0.1:38216)
0619 20:36:10.593110 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:10.593138 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:40825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260619 20:36:10.593303 26809 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.83.124:40825: Network error: Client connection negotiation failed: client connection to 127.4.83.124:40825: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.124.keytab.
Entry for principal kudu/127.4.83.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.124.keytab.
I20260619 20:36:10.606681 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.124:40825
--webserver_interface=127.4.83.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.124.keytab
--principal=kudu/127.4.83.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:37759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:37903,127.4.83.125:35755,127.4.83.124:40825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:36:10.642563 26809 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.83.124" port: 40825 } attempt: 1
I20260619 20:36:10.643090 26811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.642929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:40825 (local address 127.0.0.1:38232)
0619 20:36:10.642996 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:10.643049 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:40825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 20:36:10.643497 26811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.643399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:40825 (local address 127.0.0.1:38240)
0619 20:36:10.643444 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:10.643475 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:40825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 20:36:10.643589 26809 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.83.124:40825: Network error: Client connection negotiation failed: client connection to 127.4.83.124:40825: connect: Connection refused (error 111)
I20260619 20:36:10.651468 26734 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.83.125" port: 35755 } attempt: 3
Jun 19 20:36:10 dist-test-slave-4m58 krb5kdc[26653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901370, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for kudu/127.4.83.125@KRBTEST.COM
I20260619 20:36:10.659245 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.651871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:35755 (local address 127.0.0.1:59240)
0619 20:36:10.651930 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:10.651943 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:10.652014 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:10.652176 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:10.652178 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:10.652337 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:10.652406 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:10.652408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.653403 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.653404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:10.653963 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:10.653965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.654456 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.654457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:10.654539 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.654979 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:10.654983 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:10.656416 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:10.658251 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:10.658252 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:10.658252 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:10.658415 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:10.658656 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:10.658657 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:10.658658 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:10.658720 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:10.658995 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:10.658996 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:10.659037 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 20:36:10.659087 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 20:36:10.659174 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 20:36:10.659296 26813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.651911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59240 (local address 127.4.83.125:35755)
0619 20:36:10.651965 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.651967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.651998 (+ 31us) server_negotiation.cc:407] Connection header received
0619 20:36:10.652073 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.652074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.652089 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.652143 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:10.652508 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.653316 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.654036 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.654391 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.654460 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.656513 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:10.656514 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:10.656515 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:10.656524 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:10.658176 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:10.658507 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:10.658508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:10.658508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:10.658572 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:10.658791 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:10.658792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:10.658792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:10.658938 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:10.659002 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:10.659117 (+ 115us) server_negotiation.cc:299] Negotiation successful
0619 20:36:10.659205 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 20:36:10.659720 26734 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.124" port: 40825 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:36:10.660099 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.659996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:40825 (local address 127.0.0.1:38244)
0619 20:36:10.660043 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:10.660071 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:40825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 20:36:10.660399 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.660299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:40825 (local address 127.0.0.1:38256)
0619 20:36:10.660342 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:10.660374 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:40825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 20:36:10.660508 26734 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.83.124:40825: Network error: Client connection negotiation failed: client connection to 127.4.83.124:40825: connect: Connection refused (error 111)
W20260619 20:36:10.691385 26816 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 20:36:10.691672 26816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:10.691706 26816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:10.691736 26816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260619 20:36:10.691668 26734 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.83.124" port: 40825 } attempt: 1
I20260619 20:36:10.692152 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.692005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:40825 (local address 127.0.0.1:38262)
0619 20:36:10.692066 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:10.692113 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:40825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 20:36:10.692584 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.692476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:40825 (local address 127.0.0.1:38278)
0619 20:36:10.692525 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:10.692559 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:40825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 20:36:10.692682 26734 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.83.124:40825: Network error: Client connection negotiation failed: client connection to 127.4.83.124:40825: connect: Connection refused (error 111)
W20260619 20:36:10.693356 26816 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:10.693403 26816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:10.693423 26816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:10.693439 26816 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:10.693463 26816 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:10.695096 26816 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:37759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:37903,127.4.83.125:35755,127.4.83.124:40825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.124.keytab
--principal=kudu/127.4.83.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.124:40825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.83.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:10.695364 26816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:10.695587 26816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:10.700505 26824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:10.700577 26816 server_base.cc:1034] running on GCE node
W20260619 20:36:10.700508 26821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:10.700505 26822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:10.700949 26816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:10.701243 26816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:10.702406 26816 hybrid_clock.cc:648] HybridClock initialized: now 1781901370702390 us; error 28 us; skew 500 ppm
Jun 19 20:36:10 dist-test-slave-4m58 krb5kdc[26653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901370, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:10.704988 26816 init.cc:377] Logged in from keytab as kudu/127.4.83.124@KRBTEST.COM (short username kudu)
I20260619 20:36:10.705458 26816 webserver.cc:460] Webserver started at http://127.4.83.124:34561/ using document root <none> and password file <none>
I20260619 20:36:10.705680 26816 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:10.705725 26816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:10.705848 26816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:10.706913 26816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-2/data/instance:
uuid: "6f050896dfe64b88b282342365f54cd3"
format_stamp: "Formatted at 2026-06-19 20:36:10 on dist-test-slave-4m58"
server_key: "abb7e154bc7ec0fea2611056be79491b"
server_key_iv: "cd473c909846bdf7caf3b338b77e4f01"
server_key_version: "encryptionkey@0"
I20260619 20:36:10.707253 26816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-2/wal/instance:
uuid: "6f050896dfe64b88b282342365f54cd3"
format_stamp: "Formatted at 2026-06-19 20:36:10 on dist-test-slave-4m58"
server_key: "abb7e154bc7ec0fea2611056be79491b"
server_key_iv: "cd473c909846bdf7caf3b338b77e4f01"
server_key_version: "encryptionkey@0"
I20260619 20:36:10.708635 26816 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:36:10.709401 26831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:10.709570 26816 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:36:10.709659 26816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-2/wal
uuid: "6f050896dfe64b88b282342365f54cd3"
format_stamp: "Formatted at 2026-06-19 20:36:10 on dist-test-slave-4m58"
server_key: "abb7e154bc7ec0fea2611056be79491b"
server_key_iv: "cd473c909846bdf7caf3b338b77e4f01"
server_key_version: "encryptionkey@0"
I20260619 20:36:10.709723 26816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:10.711748 26809 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.83.124" port: 40825 } attempt: 2
I20260619 20:36:10.712234 26811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.712079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:40825 (local address 127.0.0.1:38284)
0619 20:36:10.712135 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:10.712203 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:40825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260619 20:36:10.712544 26811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.712466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:40825 (local address 127.0.0.1:38286)
0619 20:36:10.712498 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:10.712522 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:40825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260619 20:36:10.712631 26809 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.83.124:40825: Network error: Client connection negotiation failed: client connection to 127.4.83.124:40825: connect: Connection refused (error 111)
I20260619 20:36:10.721637 26816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:10.723441 26816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:10.723558 26816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:10.727383 26816 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.124:40825
I20260619 20:36:10.727422 26883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.124:40825 every 8 connection(s)
I20260619 20:36:10.727762 26816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-2/data/info.pb
I20260619 20:36:10.728821 26884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:10.730402 26884 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37903 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:36:10.731638 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 26816
I20260619 20:36:10.731741 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901102012824-4429-0/minicluster-data/master-2/wal/instance
I20260619 20:36:10.731968 4429 external_mini_cluster.cc:1428] Setting key 819dcb7e9654ead4884b3a7c94536331
Jun 19 20:36:10 dist-test-slave-4m58 krb5kdc[26653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901370, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.124@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
Jun 19 20:36:10 dist-test-slave-4m58 krb5kdc[26653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.124@KRBTEST.COM
I20260619 20:36:10.739542 26738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.731095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33294 (local address 127.4.83.126:37903)
0619 20:36:10.731150 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.731152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.731379 (+ 227us) server_negotiation.cc:407] Connection header received
0619 20:36:10.732107 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.732108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.732120 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.732182 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:10.732859 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.733337 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.734200 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.734383 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.734440 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.736803 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:10.736805 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:10.736805 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:10.736813 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:10.738072 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:10.738528 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:10.738529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:10.738530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:10.738588 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:10.738850 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:10.738850 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:10.738851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:10.738973 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:10.739027 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:10.739398 (+ 371us) server_negotiation.cc:299] Negotiation successful
0619 20:36:10.739469 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 20:36:10.739770 26886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.731082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:37903 (local address 127.0.0.1:33294)
0619 20:36:10.731257 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:10.731293 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:10.731971 (+ 678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:10.732248 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:10.732251 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:10.732567 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:10.732768 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:10.732773 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.733406 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.733407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:10.734117 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:10.734118 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.734468 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.734469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:10.734576 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.735068 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:10.735076 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:10.736723 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:10.738146 (+ 1423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:10.738154 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:10.738156 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:10.738448 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:10.738646 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:10.738647 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:10.738647 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:10.738776 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:10.739057 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:10.739062 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:10.739229 (+ 167us) client_negotiation.cc:769] Sending connection context
0619 20:36:10.739358 (+ 129us) client_negotiation.cc:240] Negotiation successful
0619 20:36:10.739521 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":94,"threads_started":1}
I20260619 20:36:10.740331 26884 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.125" port: 35755 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:36:10.740754 26888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.732343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38300 (local address 127.4.83.124:40825)
0619 20:36:10.732481 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.732485 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.732594 (+ 109us) server_negotiation.cc:407] Connection header received
0619 20:36:10.732632 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.732634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.732655 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.732715 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:10.733297 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.734077 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.734787 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.735089 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.735143 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.737626 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:10.737635 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:10.737635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:10.737641 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:10.739314 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:10.739724 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:10.739727 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:10.739727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:10.739786 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:10.740161 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:10.740162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:10.740162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:10.740321 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:10.740439 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:10.740579 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 20:36:10.740655 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":74,"threads_started":1}
Jun 19 20:36:10 dist-test-slave-4m58 krb5kdc[26653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901370, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.124@KRBTEST.COM for kudu/127.4.83.125@KRBTEST.COM
Jun 19 20:36:10 dist-test-slave-4m58 krb5kdc[26653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:36:10.747396 26813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.740661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59248 (local address 127.4.83.125:35755)
0619 20:36:10.740704 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.740705 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.740712 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:36:10.740735 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.740736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.740745 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.740791 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:10.741129 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.741838 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.742513 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.742734 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.742790 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.744706 (+ 1916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:10.744707 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:10.744708 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:10.744716 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:10.746230 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:10.746609 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:10.746611 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:10.746611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:10.746674 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:10.746891 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:10.746892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:10.746892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:10.747018 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:10.747089 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:10.747234 (+ 145us) server_negotiation.cc:299] Negotiation successful
0619 20:36:10.747303 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:36:10.747402 26886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.740589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:35755 (local address 127.0.0.1:59248)
0619 20:36:10.740633 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:10.740643 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:10.740675 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:10.740838 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:10.740838 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:10.740981 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:10.741043 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:10.741045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.741907 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.741908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:10.742398 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:10.742400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.742793 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.742795 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:10.742879 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.743256 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:10.743261 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:10.744625 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:10.746324 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:10.746325 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:10.746326 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:10.746526 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:10.746746 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:10.746747 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:10.746748 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:10.746811 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:10.747096 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:10.747097 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:10.747142 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 20:36:10.747203 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 20:36:10.747300 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 20:36:10.747709 26884 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.124" port: 40825 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:36:10.748099 26738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.741502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33310 (local address 127.4.83.126:37903)
0619 20:36:10.741536 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.741537 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.741545 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:36:10.741597 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.741598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.741605 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.741638 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:10.742130 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.742598 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.743223 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.743411 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.743462 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.745554 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:10.745555 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:10.745555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:10.745563 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:10.746906 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:10.747183 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:10.747184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:10.747184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:10.747234 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:10.747398 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:10.747399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:10.747400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:10.747526 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:10.747696 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:10.747933 (+ 237us) server_negotiation.cc:299] Negotiation successful
0619 20:36:10.748014 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
Jun 19 20:36:10 dist-test-slave-4m58 krb5kdc[26653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901370, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.124@KRBTEST.COM for kudu/127.4.83.124@KRBTEST.COM
Jun 19 20:36:10 dist-test-slave-4m58 krb5kdc[26653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.125@KRBTEST.COM
I20260619 20:36:10.755003 26886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.747979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:40825 (local address 127.0.0.1:38308)
0619 20:36:10.748069 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:10.748077 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:10.748110 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:10.748266 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:10.748267 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:10.748402 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:10.748440 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:10.748441 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.749290 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.749291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:10.750061 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:10.750063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.750549 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.750550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:10.750591 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.750964 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:10.750969 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:10.752384 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:10.754007 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:10.754008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:10.754009 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:10.754167 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:10.754350 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:10.754351 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:10.754351 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:10.754415 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:10.754736 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:10.754737 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:10.754778 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 20:36:10.754843 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 20:36:10.754931 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260619 20:36:10.755093 26888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.748041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38308 (local address 127.4.83.124:40825)
0619 20:36:10.748147 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.748149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.748157 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:36:10.748185 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.748186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.748200 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.748251 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:10.748497 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.749200 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.750149 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.750472 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.750538 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.752476 (+ 1938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:10.752477 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:10.752477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:10.752483 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:10.753936 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:10.754246 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:10.754247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:10.754248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:10.754296 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:10.754501 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:10.754502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:10.754502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:10.754634 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:10.754712 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:10.754881 (+ 169us) server_negotiation.cc:299] Negotiation successful
0619 20:36:10.754983 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":87}
I20260619 20:36:10.756335 26813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.748715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59252 (local address 127.4.83.125:35755)
0619 20:36:10.748747 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.748748 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.748755 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:36:10.748806 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.748807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.748813 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.748852 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:10.749358 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.750075 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.750861 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.751075 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.751127 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.753546 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:10.753547 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:10.753548 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:10.753556 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:10.755007 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:10.755644 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:10.755644 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:10.755645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:10.755702 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:10.755906 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:10.755907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:10.755907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:10.756013 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:10.756122 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:10.756209 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 20:36:10.756273 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 20:36:10.756899 26884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f050896dfe64b88b282342365f54cd3: Bootstrap starting.
I20260619 20:36:10.757792 4429 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:36:10.757902 26884 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f050896dfe64b88b282342365f54cd3: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:10.758174 26884 log.cc:824] T 00000000000000000000000000000000 P 6f050896dfe64b88b282342365f54cd3: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:10.758999 26884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f050896dfe64b88b282342365f54cd3: No bootstrap required, opened a new log
I20260619 20:36:10.758762 26708 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b44931b928464e7cbb7ab46f88a72f7a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33310
I20260619 20:36:10.761037 26884 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f050896dfe64b88b282342365f54cd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b44931b928464e7cbb7ab46f88a72f7a" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37903 } } peers { permanent_uuid: "c31cdc424a274057957528660c92d89d" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 35755 } } peers { permanent_uuid: "6f050896dfe64b88b282342365f54cd3" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 40825 } }
I20260619 20:36:10.761194 26884 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f050896dfe64b88b282342365f54cd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f050896dfe64b88b282342365f54cd3, State: Initialized, Role: FOLLOWER
I20260619 20:36:10.761276 26884 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f050896dfe64b88b282342365f54cd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b44931b928464e7cbb7ab46f88a72f7a" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37903 } } peers { permanent_uuid: "c31cdc424a274057957528660c92d89d" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 35755 } } peers { permanent_uuid: "6f050896dfe64b88b282342365f54cd3" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 40825 } }
I20260619 20:36:10.761549 26890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f050896dfe64b88b282342365f54cd3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b44931b928464e7cbb7ab46f88a72f7a" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37903 } } peers { permanent_uuid: "c31cdc424a274057957528660c92d89d" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 35755 } } peers { permanent_uuid: "6f050896dfe64b88b282342365f54cd3" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 40825 } } }
I20260619 20:36:10.761595 26884 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f050896dfe64b88b282342365f54cd3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:10.761646 26890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f050896dfe64b88b282342365f54cd3 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 20:36:10.763355 26901 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6f050896dfe64b88b282342365f54cd3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:36:10.763412 26901 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:36:10.763695 26816 master_runner.cc:186] Error getting master registration for 127.4.83.126:37903: OK, instance_id { permanent_uuid: "b44931b928464e7cbb7ab46f88a72f7a" instance_seqno: 1781901370422259 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 20:36:10.765784 26816 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 20:36:10.767792 26734 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.83.124" port: 40825 } attempt: 2
Jun 19 20:36:10 dist-test-slave-4m58 krb5kdc[26653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901370, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for kudu/127.4.83.124@KRBTEST.COM
I20260619 20:36:10.776007 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.768151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:40825 (local address 127.0.0.1:38324)
0619 20:36:10.768302 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:10.768316 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:10.768371 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:10.768511 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:10.768512 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:10.768702 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:10.768760 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:10.768763 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.769678 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.769679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:10.770507 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:10.770510 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.771018 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.771019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:10.771079 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.771602 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:10.771607 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:10.773158 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:10.774901 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:10.774902 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:10.774903 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:10.775124 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:10.775316 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:10.775317 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:10.775317 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:10.775373 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:10.775675 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:10.775676 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:10.775723 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 20:36:10.775782 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 20:36:10.775898 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128}
I20260619 20:36:10.776027 26888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.768243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38324 (local address 127.4.83.124:40825)
0619 20:36:10.768371 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.768373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.768392 (+ 19us) server_negotiation.cc:407] Connection header received
0619 20:36:10.768423 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.768424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.768433 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.768497 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:10.768840 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.769599 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.770613 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.770938 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.771012 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.773246 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:10.773248 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:10.773249 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:10.773257 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:10.774808 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:10.775196 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:10.775196 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:10.775197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:10.775257 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:10.775424 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:10.775425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:10.775425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:10.775612 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:10.775674 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:10.775812 (+ 138us) server_negotiation.cc:299] Negotiation successful
0619 20:36:10.775931 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 20:36:10.777659 26734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b44931b928464e7cbb7ab46f88a72f7a: Bootstrap starting.
I20260619 20:36:10.778390 26734 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b44931b928464e7cbb7ab46f88a72f7a: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:10.778690 26734 log.cc:824] T 00000000000000000000000000000000 P b44931b928464e7cbb7ab46f88a72f7a: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:10.779449 26734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b44931b928464e7cbb7ab46f88a72f7a: No bootstrap required, opened a new log
I20260619 20:36:10.780534 26734 raft_consensus.cc:348] T 00000000000000000000000000000000 P b44931b928464e7cbb7ab46f88a72f7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b44931b928464e7cbb7ab46f88a72f7a" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37903 } } peers { permanent_uuid: "c31cdc424a274057957528660c92d89d" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 35755 } } peers { permanent_uuid: "6f050896dfe64b88b282342365f54cd3" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 40825 } }
I20260619 20:36:10.780666 26734 raft_consensus.cc:729] T 00000000000000000000000000000000 P b44931b928464e7cbb7ab46f88a72f7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b44931b928464e7cbb7ab46f88a72f7a, State: Initialized, Role: FOLLOWER
I20260619 20:36:10.780743 26734 consensus_queue.cc:260] T 00000000000000000000000000000000 P b44931b928464e7cbb7ab46f88a72f7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b44931b928464e7cbb7ab46f88a72f7a" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37903 } } peers { permanent_uuid: "c31cdc424a274057957528660c92d89d" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 35755 } } peers { permanent_uuid: "6f050896dfe64b88b282342365f54cd3" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 40825 } }
I20260619 20:36:10.781076 26734 sys_catalog.cc:564] T 00000000000000000000000000000000 P b44931b928464e7cbb7ab46f88a72f7a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:10.781030 26904 sys_catalog.cc:455] T 00000000000000000000000000000000 P b44931b928464e7cbb7ab46f88a72f7a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b44931b928464e7cbb7ab46f88a72f7a" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37903 } } peers { permanent_uuid: "c31cdc424a274057957528660c92d89d" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 35755 } } peers { permanent_uuid: "6f050896dfe64b88b282342365f54cd3" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 40825 } } }
I20260619 20:36:10.781150 26904 sys_catalog.cc:458] T 00000000000000000000000000000000 P b44931b928464e7cbb7ab46f88a72f7a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:36:10.781098 26708 raft_consensus.cc:482] T 00000000000000000000000000000000 P b44931b928464e7cbb7ab46f88a72f7a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 20:36:10.781251 26708 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b44931b928464e7cbb7ab46f88a72f7a [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:10.782081 26708 raft_consensus.cc:504] T 00000000000000000000000000000000 P b44931b928464e7cbb7ab46f88a72f7a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b44931b928464e7cbb7ab46f88a72f7a" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37903 } } peers { permanent_uuid: "c31cdc424a274057957528660c92d89d" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 35755 } } peers { permanent_uuid: "6f050896dfe64b88b282342365f54cd3" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 40825 } }
I20260619 20:36:10.782359 26708 leader_election.cc:288] T 00000000000000000000000000000000 P b44931b928464e7cbb7ab46f88a72f7a [CANDIDATE]: Term 1 election: Requested vote from peers c31cdc424a274057957528660c92d89d (127.4.83.125:35755), 6f050896dfe64b88b282342365f54cd3 (127.4.83.124:40825)
I20260619 20:36:10.782946 26858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b44931b928464e7cbb7ab46f88a72f7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6f050896dfe64b88b282342365f54cd3"
I20260619 20:36:10.783097 26858 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f050896dfe64b88b282342365f54cd3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:10.784053 26858 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6f050896dfe64b88b282342365f54cd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b44931b928464e7cbb7ab46f88a72f7a in term 1.
I20260619 20:36:10.784324 26683 leader_election.cc:302] T 00000000000000000000000000000000 P b44931b928464e7cbb7ab46f88a72f7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f050896dfe64b88b282342365f54cd3, b44931b928464e7cbb7ab46f88a72f7a; no voters:
I20260619 20:36:10.784446 26904 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b44931b928464e7cbb7ab46f88a72f7a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:10.784588 26904 raft_consensus.cc:686] T 00000000000000000000000000000000 P b44931b928464e7cbb7ab46f88a72f7a [term 1 LEADER]: Becoming Leader. State: Replica: b44931b928464e7cbb7ab46f88a72f7a, State: Running, Role: LEADER
I20260619 20:36:10.784682 26904 consensus_queue.cc:237] T 00000000000000000000000000000000 P b44931b928464e7cbb7ab46f88a72f7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b44931b928464e7cbb7ab46f88a72f7a" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37903 } } peers { permanent_uuid: "c31cdc424a274057957528660c92d89d" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 35755 } } peers { permanent_uuid: "6f050896dfe64b88b282342365f54cd3" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 40825 } }
I20260619 20:36:10.785084 26904 sys_catalog.cc:455] T 00000000000000000000000000000000 P b44931b928464e7cbb7ab46f88a72f7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b44931b928464e7cbb7ab46f88a72f7a. Latest consensus state: current_term: 1 leader_uuid: "b44931b928464e7cbb7ab46f88a72f7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b44931b928464e7cbb7ab46f88a72f7a" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37903 } } peers { permanent_uuid: "c31cdc424a274057957528660c92d89d" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 35755 } } peers { permanent_uuid: "6f050896dfe64b88b282342365f54cd3" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 40825 } } }
I20260619 20:36:10.785156 26904 sys_catalog.cc:458] T 00000000000000000000000000000000 P b44931b928464e7cbb7ab46f88a72f7a [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:10.785372 26925 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:36:10.785868 26925 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:36:10.783788 26666 master_runner.cc:186] Error getting master registration for 127.4.83.125:35755: OK, instance_id { permanent_uuid: "c31cdc424a274057957528660c92d89d" instance_seqno: 1781901370565906 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 20:36:10.786873 26666 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 20:36:10.790652 26858 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6f050896dfe64b88b282342365f54cd3 [term 1 FOLLOWER]: Refusing update from remote peer b44931b928464e7cbb7ab46f88a72f7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:10.791028 26904 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b44931b928464e7cbb7ab46f88a72f7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f050896dfe64b88b282342365f54cd3" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 40825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:10.793584 26890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f050896dfe64b88b282342365f54cd3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b44931b928464e7cbb7ab46f88a72f7a. Latest consensus state: current_term: 1 leader_uuid: "b44931b928464e7cbb7ab46f88a72f7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b44931b928464e7cbb7ab46f88a72f7a" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37903 } } peers { permanent_uuid: "c31cdc424a274057957528660c92d89d" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 35755 } } peers { permanent_uuid: "6f050896dfe64b88b282342365f54cd3" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 40825 } } }
I20260619 20:36:10.793689 26890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f050896dfe64b88b282342365f54cd3 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:36:10.796144 26738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.783517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33314 (local address 127.4.83.126:37903)
0619 20:36:10.783553 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.783555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.784817 (+ 1262us) server_negotiation.cc:407] Connection header received
0619 20:36:10.785083 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.785084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.785096 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.785126 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:10.786481 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.787182 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.788475 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.788748 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.788815 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.791914 (+ 3099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:10.791916 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:10.791916 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:10.791925 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:10.793732 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:10.794245 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:10.794246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:10.794247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:10.794309 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:10.794530 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:10.794531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:10.794531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:10.794687 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:10.794789 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:10.795943 (+ 1154us) server_negotiation.cc:299] Negotiation successful
0619 20:36:10.796030 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 20:36:10.795691 26890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f050896dfe64b88b282342365f54cd3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b44931b928464e7cbb7ab46f88a72f7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b44931b928464e7cbb7ab46f88a72f7a" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37903 } } peers { permanent_uuid: "c31cdc424a274057957528660c92d89d" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 35755 } } peers { permanent_uuid: "6f050896dfe64b88b282342365f54cd3" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 40825 } } }
I20260619 20:36:10.796306 26890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f050896dfe64b88b282342365f54cd3 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:36:10.797742 26928 mvcc.cc:205] Tried to move back new op lower bound from 7298668014750695424 to 7298668014734827520. Current Snapshot: MvccSnapshot[applied={T|T < 7298668014750695424}]
I20260619 20:36:10.795727 26927 mvcc.cc:205] Tried to move back new op lower bound from 7298668014750695424 to 7298668014734827520. Current Snapshot: MvccSnapshot[applied={T|T < 7298668014750695424}]
I20260619 20:36:10.798975 26923 sys_catalog.cc:455] T 00000000000000000000000000000000 P b44931b928464e7cbb7ab46f88a72f7a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b44931b928464e7cbb7ab46f88a72f7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b44931b928464e7cbb7ab46f88a72f7a" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37903 } } peers { permanent_uuid: "c31cdc424a274057957528660c92d89d" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 35755 } } peers { permanent_uuid: "6f050896dfe64b88b282342365f54cd3" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 40825 } } }
I20260619 20:36:10.799084 26923 sys_catalog.cc:458] T 00000000000000000000000000000000 P b44931b928464e7cbb7ab46f88a72f7a [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:10.804003 26809 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.83.124" port: 40825 } attempt: 3
I20260619 20:36:10.804376 26888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.787015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38328 (local address 127.4.83.124:40825)
0619 20:36:10.787053 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.787055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.788037 (+ 982us) server_negotiation.cc:407] Connection header received
0619 20:36:10.788171 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.788172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.788183 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.788210 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:10.794708 (+ 6498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.795409 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.797294 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.797569 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.797631 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.799645 (+ 2014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:10.799646 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:10.799647 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:10.799654 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:10.802629 (+ 2975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:10.803436 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:10.803438 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:10.803438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:10.803505 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:10.803716 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:10.803717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:10.803718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:10.803883 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:10.803971 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:10.804189 (+ 218us) server_negotiation.cc:299] Negotiation successful
0619 20:36:10.804281 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:36:10.808336 26813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.783968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59264 (local address 127.4.83.125:35755)
0619 20:36:10.784006 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.784008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.786706 (+ 2698us) server_negotiation.cc:407] Connection header received
0619 20:36:10.786843 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.786844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.786855 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.786883 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:10.794278 (+ 7395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.795022 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.797723 (+ 2701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.798031 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.798072 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.800071 (+ 1999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:10.800072 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:10.800073 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:10.800081 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:10.801619 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:10.802884 (+ 1265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:10.802886 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:10.802886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:10.803217 (+ 331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:10.803708 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:10.803709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:10.803709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:10.803883 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:10.803977 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:10.808117 (+ 4140us) server_negotiation.cc:299] Negotiation successful
0619 20:36:10.808241 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:36:10.808954 26925 catalog_manager.cc:1347] Generated new cluster ID: 5e8cdb86c4f74fa29d1811d55cecac7a
I20260619 20:36:10.809019 26925 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:36:10 dist-test-slave-4m58 krb5kdc[26653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901370, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.125@KRBTEST.COM for kudu/127.4.83.124@KRBTEST.COM
I20260619 20:36:10.813133 26811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.804302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:40825 (local address 127.0.0.1:38340)
0619 20:36:10.804536 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:10.804548 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:10.804597 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:10.804751 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:10.804752 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:10.804917 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:10.804975 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:10.804977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.805917 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.805919 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:10.806871 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:10.806875 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.807502 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.807503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:10.807562 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.808079 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:10.808085 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:10.809691 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:10.811942 (+ 2251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:10.811944 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:10.811944 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:10.812162 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:10.812436 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:10.812437 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:10.812437 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:10.812518 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:10.812812 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:10.812813 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:10.812864 (+ 51us) client_negotiation.cc:769] Sending connection context
0619 20:36:10.812925 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 20:36:10.813031 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216}
I20260619 20:36:10.813133 26888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.804385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38340 (local address 127.4.83.124:40825)
0619 20:36:10.804497 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.804498 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.804649 (+ 151us) server_negotiation.cc:407] Connection header received
0619 20:36:10.804671 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.804672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.804682 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.805042 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:10.805076 (+ 34us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.805805 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.807002 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.807421 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.807499 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.809790 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:10.809792 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:10.809792 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:10.809799 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:10.811786 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:10.812298 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:10.812299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:10.812300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:10.812383 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:10.812595 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:10.812596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:10.812596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:10.812751 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:10.812809 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:10.812950 (+ 141us) server_negotiation.cc:299] Negotiation successful
0619 20:36:10.813037 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":114,"threads_started":1}
I20260619 20:36:10.815177 26809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c31cdc424a274057957528660c92d89d: Bootstrap starting.
I20260619 20:36:10.815977 26809 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c31cdc424a274057957528660c92d89d: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:10.816290 26809 log.cc:824] T 00000000000000000000000000000000 P c31cdc424a274057957528660c92d89d: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:10.828917 26809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c31cdc424a274057957528660c92d89d: No bootstrap required, opened a new log
I20260619 20:36:10.831022 26925 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:36:10.831810 26925 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:36:10.832448 26809 raft_consensus.cc:348] T 00000000000000000000000000000000 P c31cdc424a274057957528660c92d89d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b44931b928464e7cbb7ab46f88a72f7a" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37903 } } peers { permanent_uuid: "c31cdc424a274057957528660c92d89d" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 35755 } } peers { permanent_uuid: "6f050896dfe64b88b282342365f54cd3" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 40825 } }
I20260619 20:36:10.832715 26809 raft_consensus.cc:729] T 00000000000000000000000000000000 P c31cdc424a274057957528660c92d89d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c31cdc424a274057957528660c92d89d, State: Initialized, Role: FOLLOWER
I20260619 20:36:10.832855 26809 consensus_queue.cc:260] T 00000000000000000000000000000000 P c31cdc424a274057957528660c92d89d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b44931b928464e7cbb7ab46f88a72f7a" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37903 } } peers { permanent_uuid: "c31cdc424a274057957528660c92d89d" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 35755 } } peers { permanent_uuid: "6f050896dfe64b88b282342365f54cd3" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 40825 } }
I20260619 20:36:10.833578 26783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b44931b928464e7cbb7ab46f88a72f7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c31cdc424a274057957528660c92d89d"
I20260619 20:36:10.833678 26783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c31cdc424a274057957528660c92d89d [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:10.834616 26783 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c31cdc424a274057957528660c92d89d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b44931b928464e7cbb7ab46f88a72f7a in term 1.
I20260619 20:36:10.834908 26925 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b44931b928464e7cbb7ab46f88a72f7a: Generated new TSK 0
I20260619 20:36:10.835099 26925 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:36:10.836416 26782 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c31cdc424a274057957528660c92d89d [term 1 FOLLOWER]: Refusing update from remote peer b44931b928464e7cbb7ab46f88a72f7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:10.836688 26932 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b44931b928464e7cbb7ab46f88a72f7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c31cdc424a274057957528660c92d89d" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 35755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:10.839017 26936 sys_catalog.cc:455] T 00000000000000000000000000000000 P c31cdc424a274057957528660c92d89d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b44931b928464e7cbb7ab46f88a72f7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b44931b928464e7cbb7ab46f88a72f7a" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37903 } } peers { permanent_uuid: "c31cdc424a274057957528660c92d89d" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 35755 } } peers { permanent_uuid: "6f050896dfe64b88b282342365f54cd3" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 40825 } } }
I20260619 20:36:10.839129 26936 sys_catalog.cc:458] T 00000000000000000000000000000000 P c31cdc424a274057957528660c92d89d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:36:10.839027 26937 mvcc.cc:205] Tried to move back new op lower bound from 7298668014935388160 to 7298668014734827520. Current Snapshot: MvccSnapshot[applied={T|T < 7298668014750695424}]
I20260619 20:36:10.839284 26936 sys_catalog.cc:455] T 00000000000000000000000000000000 P c31cdc424a274057957528660c92d89d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b44931b928464e7cbb7ab46f88a72f7a. Latest consensus state: current_term: 1 leader_uuid: "b44931b928464e7cbb7ab46f88a72f7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b44931b928464e7cbb7ab46f88a72f7a" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37903 } } peers { permanent_uuid: "c31cdc424a274057957528660c92d89d" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 35755 } } peers { permanent_uuid: "6f050896dfe64b88b282342365f54cd3" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 40825 } } }
I20260619 20:36:10.839337 26936 sys_catalog.cc:458] T 00000000000000000000000000000000 P c31cdc424a274057957528660c92d89d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:36:10.839367 26936 sys_catalog.cc:455] T 00000000000000000000000000000000 P c31cdc424a274057957528660c92d89d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b44931b928464e7cbb7ab46f88a72f7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b44931b928464e7cbb7ab46f88a72f7a" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37903 } } peers { permanent_uuid: "c31cdc424a274057957528660c92d89d" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 35755 } } peers { permanent_uuid: "6f050896dfe64b88b282342365f54cd3" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 40825 } } }
I20260619 20:36:10.839408 26936 sys_catalog.cc:458] T 00000000000000000000000000000000 P c31cdc424a274057957528660c92d89d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:36:10.833516 26809 sys_catalog.cc:564] T 00000000000000000000000000000000 P c31cdc424a274057957528660c92d89d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:10.841327 26953 catalog_manager.cc:1259] Loaded cluster ID: 5e8cdb86c4f74fa29d1811d55cecac7a
I20260619 20:36:10.841382 26953 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c31cdc424a274057957528660c92d89d: loading cluster ID for follower catalog manager: success
I20260619 20:36:10.842549 26953 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c31cdc424a274057957528660c92d89d: acquiring CA information for follower catalog manager: success
I20260619 20:36:10.842864 26953 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c31cdc424a274057957528660c92d89d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 20:36:10.867266 26738 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:10.865666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33322 (local address 127.4.83.126:37903)
0619 20:36:10.865713 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.865715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.866551 (+ 836us) server_negotiation.cc:407] Connection header received
0619 20:36:10.866650 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.866652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.866663 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.866695 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:10.867185 (+ 490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33322: BlockingRecv error: recv got EOF from 127.0.0.1:33322 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 20:36:10.868044 26813 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:10.865995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59270 (local address 127.4.83.125:35755)
0619 20:36:10.866039 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.866040 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.867512 (+ 1472us) server_negotiation.cc:407] Connection header received
0619 20:36:10.867606 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.867607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.867619 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.867647 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:10.867982 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59270: BlockingRecv error: recv got EOF from 127.0.0.1:59270 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 20:36:10.868994 26888 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:10.866291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38346 (local address 127.4.83.124:40825)
0619 20:36:10.866339 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.866341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.868385 (+ 2044us) server_negotiation.cc:407] Connection header received
0619 20:36:10.868451 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.868452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.868464 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.868512 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:10.868928 (+ 416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38346: BlockingRecv error: recv got EOF from 127.0.0.1:38346 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 20:36:10.874820 26813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.870435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59284 (local address 127.4.83.125:35755)
0619 20:36:10.870473 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.870475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.870704 (+ 229us) server_negotiation.cc:407] Connection header received
0619 20:36:10.870814 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.870815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.870860 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.870902 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:10.871096 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.872262 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.873686 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.874090 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.874162 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.874398 (+ 236us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:10.874540 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:10.874589 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:10.874700 (+ 111us) server_negotiation.cc:299] Negotiation successful
0619 20:36:10.874727 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 20:36:10.878427 26888 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:10.877475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38350 (local address 127.4.83.124:40825)
0619 20:36:10.877523 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.877525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.877794 (+ 269us) server_negotiation.cc:407] Connection header received
0619 20:36:10.877907 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.877908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.877931 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.877960 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:10.878364 (+ 404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38350: BlockingRecv error: recv got EOF from 127.0.0.1:38350 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:36:10.884438 26813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.880951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59294 (local address 127.4.83.125:35755)
0619 20:36:10.880999 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.881001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.881261 (+ 260us) server_negotiation.cc:407] Connection header received
0619 20:36:10.881361 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.881362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.881373 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.881401 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:10.881593 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.882373 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.883570 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.883906 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.883959 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.884166 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:10.884204 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:10.884230 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:10.884326 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 20:36:10.884349 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260619 20:36:10.887799 26888 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:10.886937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38362 (local address 127.4.83.124:40825)
0619 20:36:10.886984 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.886986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.887218 (+ 232us) server_negotiation.cc:407] Connection header received
0619 20:36:10.887315 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.887316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.887328 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.887357 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:10.887731 (+ 374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38362: BlockingRecv error: recv got EOF from 127.0.0.1:38362 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:36:10.894447 26813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.891141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59310 (local address 127.4.83.125:35755)
0619 20:36:10.891185 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.891188 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.891413 (+ 225us) server_negotiation.cc:407] Connection header received
0619 20:36:10.891534 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.891535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.891545 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.891571 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:10.891732 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.892565 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.893657 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.893959 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.893998 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.894187 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:10.894224 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:10.894250 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:10.894344 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 20:36:10.894367 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 20:36:10.897703 26888 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:10.896839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38364 (local address 127.4.83.124:40825)
0619 20:36:10.896887 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.896888 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.897165 (+ 277us) server_negotiation.cc:407] Connection header received
0619 20:36:10.897258 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.897259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.897272 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.897299 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:10.897643 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38364: BlockingRecv error: recv got EOF from 127.0.0.1:38364 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:36:10.906667 26813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.903318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59316 (local address 127.4.83.125:35755)
0619 20:36:10.903365 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.903367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.903656 (+ 289us) server_negotiation.cc:407] Connection header received
0619 20:36:10.903755 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.903756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.903766 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.903792 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:10.904045 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.904756 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.905845 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.906131 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.906174 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.906381 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:10.906416 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:10.906443 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:10.906558 (+ 115us) server_negotiation.cc:299] Negotiation successful
0619 20:36:10.906581 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 20:36:10.910074 26888 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:10.909221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38380 (local address 127.4.83.124:40825)
0619 20:36:10.909264 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.909266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.909476 (+ 210us) server_negotiation.cc:407] Connection header received
0619 20:36:10.909575 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.909576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.909589 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.909617 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:10.910008 (+ 391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38380: BlockingRecv error: recv got EOF from 127.0.0.1:38380 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:36:10.923032 26813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.919494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59332 (local address 127.4.83.125:35755)
0619 20:36:10.919556 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.919558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.919887 (+ 329us) server_negotiation.cc:407] Connection header received
0619 20:36:10.920019 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.920020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.920033 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.920062 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:10.920264 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.921012 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.922175 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.922467 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.922513 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.922750 (+ 237us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:10.922788 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:10.922814 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:10.922915 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 20:36:10.922949 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 20:36:10.926405 26888 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:10.925561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38384 (local address 127.4.83.124:40825)
0619 20:36:10.925607 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.925608 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.925823 (+ 215us) server_negotiation.cc:407] Connection header received
0619 20:36:10.925918 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.925919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.925931 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.925971 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:10.926345 (+ 374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38384: BlockingRecv error: recv got EOF from 127.0.0.1:38384 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:36:10.947196 26813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.943822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59340 (local address 127.4.83.125:35755)
0619 20:36:10.943891 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.943894 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.944201 (+ 307us) server_negotiation.cc:407] Connection header received
0619 20:36:10.944305 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.944306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.944318 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.944346 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:10.944535 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.945274 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.946395 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.946665 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.946707 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.946906 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:10.946960 (+ 54us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:10.946986 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:10.947089 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 20:36:10.947112 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260619 20:36:10.950466 26888 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:10.949576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38400 (local address 127.4.83.124:40825)
0619 20:36:10.949618 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.949620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.949874 (+ 254us) server_negotiation.cc:407] Connection header received
0619 20:36:10.949992 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.949993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.950006 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.950035 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:10.950402 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38400: BlockingRecv error: recv got EOF from 127.0.0.1:38400 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:36:10.986366 26813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:10.983714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59344 (local address 127.4.83.125:35755)
0619 20:36:10.983769 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.983772 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.983845 (+ 73us) server_negotiation.cc:407] Connection header received
0619 20:36:10.983947 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.983948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.983966 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.984047 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:10.984146 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.984850 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.985643 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:10.985929 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:10.986016 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:10.986139 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:10.986174 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:10.986226 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:10.986262 (+ 36us) server_negotiation.cc:299] Negotiation successful
0619 20:36:10.986282 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
W20260619 20:36:10.988770 26888 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:10.987963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38408 (local address 127.4.83.124:40825)
0619 20:36:10.988024 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:10.988026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:10.988221 (+ 195us) server_negotiation.cc:407] Connection header received
0619 20:36:10.988266 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:10.988267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:10.988278 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:10.988333 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:10.988717 (+ 384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38408: BlockingRecv error: recv got EOF from 127.0.0.1:38408 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 20:36:11.056200 26813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:11.053849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59346 (local address 127.4.83.125:35755)
0619 20:36:11.053935 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:11.053937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:11.054017 (+ 80us) server_negotiation.cc:407] Connection header received
0619 20:36:11.054103 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:11.054104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:11.054115 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:11.054157 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:11.054272 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:11.054761 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:11.055537 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:11.055740 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:11.055796 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:11.055954 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:11.055997 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:11.056038 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:11.056099 (+ 61us) server_negotiation.cc:299] Negotiation successful
0619 20:36:11.056118 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260619 20:36:11.058209 26888 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:11.057691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38420 (local address 127.4.83.124:40825)
0619 20:36:11.057737 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:11.057739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:11.057787 (+ 48us) server_negotiation.cc:407] Connection header received
0619 20:36:11.057839 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:11.057840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:11.057849 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:11.057890 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:11.058154 (+ 264us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38420: BlockingRecv error: recv got EOF from 127.0.0.1:38420 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:36:11.189642 26813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:11.187363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59354 (local address 127.4.83.125:35755)
0619 20:36:11.187417 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:11.187419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:11.187535 (+ 116us) server_negotiation.cc:407] Connection header received
0619 20:36:11.187575 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:11.187576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:11.187586 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:11.187632 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:11.187797 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:11.188352 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:11.189103 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:11.189290 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:11.189349 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:11.189439 (+ 90us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:11.189493 (+ 54us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:11.189527 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:11.189565 (+ 38us) server_negotiation.cc:299] Negotiation successful
0619 20:36:11.189580 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260619 20:36:11.191779 26888 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:11.191230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38434 (local address 127.4.83.124:40825)
0619 20:36:11.191297 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:11.191299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:11.191308 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:36:11.191364 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:11.191365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:11.191376 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:11.191434 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:11.191716 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38434: BlockingRecv error: recv got EOF from 127.0.0.1:38434 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 20:36:11.217993 26923 sys_catalog.cc:455] T 00000000000000000000000000000000 P b44931b928464e7cbb7ab46f88a72f7a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b44931b928464e7cbb7ab46f88a72f7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b44931b928464e7cbb7ab46f88a72f7a" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 37903 } } peers { permanent_uuid: "c31cdc424a274057957528660c92d89d" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 35755 } } peers { permanent_uuid: "6f050896dfe64b88b282342365f54cd3" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 40825 } } }
I20260619 20:36:11.218166 26923 sys_catalog.cc:458] T 00000000000000000000000000000000 P b44931b928464e7cbb7ab46f88a72f7a [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:11.451926 26813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:11.449335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59370 (local address 127.4.83.125:35755)
0619 20:36:11.449411 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:11.449413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:11.449429 (+ 16us) server_negotiation.cc:407] Connection header received
0619 20:36:11.449539 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:11.449540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:11.449552 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:11.449615 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:11.449817 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:11.450365 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:11.451205 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:11.451407 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:11.451469 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:11.451658 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:11.451694 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:11.451735 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:11.451783 (+ 48us) server_negotiation.cc:299] Negotiation successful
0619 20:36:11.451805 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260619 20:36:11.454696 26888 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:11.454027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38436 (local address 127.4.83.124:40825)
0619 20:36:11.454101 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:11.454104 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:11.454113 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:36:11.454146 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:11.454147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:11.454158 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:11.454218 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:11.454612 (+ 394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38436: BlockingRecv error: recv got EOF from 127.0.0.1:38436 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 20:36:11.764068 26901 catalog_manager.cc:1259] Loaded cluster ID: 5e8cdb86c4f74fa29d1811d55cecac7a
I20260619 20:36:11.764175 26901 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6f050896dfe64b88b282342365f54cd3: loading cluster ID for follower catalog manager: success
I20260619 20:36:11.765080 26901 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6f050896dfe64b88b282342365f54cd3: acquiring CA information for follower catalog manager: success
I20260619 20:36:11.765435 26901 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6f050896dfe64b88b282342365f54cd3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 20:36:11.970996 27069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:11.968402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59372 (local address 127.4.83.125:35755)
0619 20:36:11.968544 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:11.968546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:11.968559 (+ 13us) server_negotiation.cc:407] Connection header received
0619 20:36:11.968651 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:11.968652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:11.968669 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:11.968730 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:11.968892 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:11.969481 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:11.970303 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:11.970516 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:11.970572 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:11.970707 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:11.970748 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:11.970790 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:11.970856 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 20:36:11.970883 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":66,"threads_started":1}
I20260619 20:36:11.975507 27075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:11.972889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38444 (local address 127.4.83.124:40825)
0619 20:36:11.972999 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:11.973001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:11.973011 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:36:11.973039 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:11.973039 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:11.973105 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:11.973188 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:11.973348 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:11.973912 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:11.974701 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:11.974911 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:11.974953 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:11.975083 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:11.975254 (+ 171us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:11.975341 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:11.975370 (+ 29us) server_negotiation.cc:299] Negotiation successful
0619 20:36:11.975398 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":51,"threads_started":1}
I20260619 20:36:11.980484 27075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:11.977717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38446 (local address 127.4.83.124:40825)
0619 20:36:11.977763 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:11.977765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:11.977800 (+ 35us) server_negotiation.cc:407] Connection header received
0619 20:36:11.977887 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:11.977888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:11.977902 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:11.977953 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:11.978063 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:11.978583 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:11.979625 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:11.979965 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:11.980032 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:11.980170 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:11.980213 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:11.980257 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:11.980344 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 20:36:11.980380 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:36:11.980695 27069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:11.977585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59382 (local address 127.4.83.125:35755)
0619 20:36:11.977633 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:11.977635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:11.977815 (+ 180us) server_negotiation.cc:407] Connection header received
0619 20:36:11.978014 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:11.978015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:11.978033 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:11.978081 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:11.978257 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:11.979001 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:11.979809 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:11.980117 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:11.980156 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:11.980366 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:11.980404 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:11.980430 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:11.980575 (+ 145us) server_negotiation.cc:299] Negotiation successful
0619 20:36:11.980609 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:36:11.980983 27081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:11.977451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33338 (local address 127.4.83.126:37903)
0619 20:36:11.977656 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:11.977657 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:11.977666 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:36:11.977725 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:11.977726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:11.977801 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:11.977921 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:11.978033 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:11.978856 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:11.979907 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:11.980201 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:11.980256 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:11.980479 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:11.980676 (+ 197us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:11.980775 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:11.980856 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 20:36:11.980890 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":64,"threads_started":1}
I20260619 20:36:11.982785 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 26666
I20260619 20:36:11.988252 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 26741
I20260619 20:36:11.993520 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 26816
2026-06-19T20:36:11Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2782 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:36:12 dist-test-slave-4m58 krb5kdc[27086](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:36:12 dist-test-slave-4m58 krb5kdc[27086](info): set up 2 sockets
Jun 19 20:36:12 dist-test-slave-4m58 krb5kdc[27086](info): commencing operation
krb5kdc: starting...
W20260619 20:36:12.968858 4429 mini_kdc.cc:121] Time spent starting KDC: real 0.954s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/krb5kdc/test-user.keytab.
Jun 19 20:36:13 dist-test-slave-4m58 krb5kdc[27086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:36:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:36:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.83.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
Entry for principal kudu/127.4.83.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab.
I20260619 20:36:13.087900 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:45623
--webserver_interface=127.4.83.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:44447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:45623,127.4.83.125:34851,127.4.83.124:46801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:13.175518 27099 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 20:36:13.175869 27099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:13.175915 27099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:13.175957 27099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:13.177332 27099 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:13.177386 27099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:13.177407 27099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:13.177423 27099 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:13.177438 27099 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:13.179158 27099 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:44447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:45623,127.4.83.125:34851,127.4.83.124:46801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.126.keytab
--principal=kudu/127.4.83.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.126:45623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.83.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:13.179438 27099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:13.179672 27099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:13.184480 27105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:13.184512 27107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:13.184489 27104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:13.184849 27099 server_base.cc:1034] running on GCE node
I20260619 20:36:13.185096 27099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:13.185387 27099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:13.186563 27099 hybrid_clock.cc:648] HybridClock initialized: now 1781901373186547 us; error 30 us; skew 500 ppm
Jun 19 20:36:13 dist-test-slave-4m58 krb5kdc[27086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901373, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:13.189926 27099 init.cc:377] Logged in from keytab as kudu/127.4.83.126@KRBTEST.COM (short username kudu)
I20260619 20:36:13.190423 27099 webserver.cc:460] Webserver started at http://127.4.83.126:41169/ using document root <none> and password file <none>
I20260619 20:36:13.190653 27099 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:13.190702 27099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:13.190834 27099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:13.192107 27099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-0/data/instance:
uuid: "b6e95c8bd7dc4d73b3ca73485d942950"
format_stamp: "Formatted at 2026-06-19 20:36:13 on dist-test-slave-4m58"
server_key: "28e3292f540a8e6adc4d9e170d43aef4"
server_key_iv: "3d18760630dbc0625c41e13f2ea8ac11"
server_key_version: "encryptionkey@0"
I20260619 20:36:13.192479 27099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-0/wal/instance:
uuid: "b6e95c8bd7dc4d73b3ca73485d942950"
format_stamp: "Formatted at 2026-06-19 20:36:13 on dist-test-slave-4m58"
server_key: "28e3292f540a8e6adc4d9e170d43aef4"
server_key_iv: "3d18760630dbc0625c41e13f2ea8ac11"
server_key_version: "encryptionkey@0"
I20260619 20:36:13.193919 27099 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 20:36:13.194753 27114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:13.194972 27099 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:36:13.195087 27099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-0/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-0/wal
uuid: "b6e95c8bd7dc4d73b3ca73485d942950"
format_stamp: "Formatted at 2026-06-19 20:36:13 on dist-test-slave-4m58"
server_key: "28e3292f540a8e6adc4d9e170d43aef4"
server_key_iv: "3d18760630dbc0625c41e13f2ea8ac11"
server_key_version: "encryptionkey@0"
I20260619 20:36:13.195158 27099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:13.206200 27099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:13.209354 27099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:13.209518 27099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:13.213575 27099 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.126:45623
I20260619 20:36:13.213616 27166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.126:45623 every 8 connection(s)
I20260619 20:36:13.213979 27099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-0/data/info.pb
I20260619 20:36:13.214946 27167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:13.216668 27167 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45623 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 20:36:13 dist-test-slave-4m58 krb5kdc[27086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901373, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:36:13.223505 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 27099
I20260619 20:36:13.223604 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-0/wal/instance
I20260619 20:36:13.223791 4429 external_mini_cluster.cc:1428] Setting key 02c903057e20a440f667b43d276984de
I20260619 20:36:13.226600 27169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.217340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:45623 (local address 127.0.0.1:42226)
0619 20:36:13.217641 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:13.217682 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:13.218287 (+ 605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:13.218627 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:13.218630 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:13.218907 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:13.219088 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:13.219092 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.220279 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.220280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:13.221002 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:13.221005 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.221510 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.221511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:13.221572 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:13.222101 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:13.222109 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:13.223571 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:13.225208 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:13.225214 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:13.225216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:13.225420 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:13.225634 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:13.225635 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:13.225635 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:13.225713 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:13.226046 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:13.226051 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:13.226114 (+ 63us) client_negotiation.cc:769] Sending connection context
0619 20:36:13.226195 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 20:36:13.226413 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":77,"thread_start_us":113,"threads_started":1}
I20260619 20:36:13.226603 27170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.217392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42226 (local address 127.4.83.126:45623)
0619 20:36:13.217574 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:13.217585 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:13.217810 (+ 225us) server_negotiation.cc:407] Connection header received
0619 20:36:13.218482 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:13.218484 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:13.218502 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:13.218590 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:13.219201 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.220179 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.221082 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.221393 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.221482 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:13.223670 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:13.223678 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:13.223678 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:13.223685 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:13.225131 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:13.225498 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:13.225501 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:13.225501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:13.225576 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:13.225765 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:13.225765 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:13.225765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:13.225986 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:13.226056 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:13.226235 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 20:36:13.226344 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":78,"threads_started":1}
I20260619 20:36:13.227499 27167 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.125" port: 34851 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:36:13.227934 27169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.227738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:34851 (local address 127.0.0.1:33466)
0619 20:36:13.227802 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:13.227904 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.125:34851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260619 20:36:13.228076 27116 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.83.125:34851: connect: Connection refused (error 111)
I20260619 20:36:13.228536 27169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.228413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:34851 (local address 127.0.0.1:33478)
0619 20:36:13.228460 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:13.228486 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.125:34851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260619 20:36:13.228636 27167 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.83.125:34851: Network error: Client connection negotiation failed: client connection to 127.4.83.125:34851: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.4.83.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.125.keytab.
Entry for principal kudu/127.4.83.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.125.keytab.
I20260619 20:36:13.248610 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.125:34851
--webserver_interface=127.4.83.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.125.keytab
--principal=kudu/127.4.83.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:44447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:45623,127.4.83.125:34851,127.4.83.124:46801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:36:13.277884 27167 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.83.125" port: 34851 } attempt: 1
I20260619 20:36:13.278440 27169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.278296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:34851 (local address 127.0.0.1:33488)
0619 20:36:13.278365 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:13.278408 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.125:34851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 20:36:13.278945 27169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.278829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:34851 (local address 127.0.0.1:33492)
0619 20:36:13.278894 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:13.278920 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.125:34851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260619 20:36:13.279040 27167 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.83.125:34851: Network error: Client connection negotiation failed: client connection to 127.4.83.125:34851: connect: Connection refused (error 111)
W20260619 20:36:13.330271 27174 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 20:36:13.330536 27174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:13.330572 27174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:13.330602 27174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:13.332134 27174 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:13.332198 27174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:13.332218 27174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:13.332234 27174 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:13.332249 27174 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:13.333966 27174 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:44447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:45623,127.4.83.125:34851,127.4.83.124:46801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.125.keytab
--principal=kudu/127.4.83.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.125:34851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.83.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:13.334254 27174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:13.334487 27174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:13.339341 27182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:13.339341 27180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:13.339414 27179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:13.339964 27174 server_base.cc:1034] running on GCE node
I20260619 20:36:13.340235 27174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:13.340562 27174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:13.341725 27174 hybrid_clock.cc:648] HybridClock initialized: now 1781901373341702 us; error 36 us; skew 500 ppm
Jun 19 20:36:13 dist-test-slave-4m58 krb5kdc[27086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901373, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:13.344838 27174 init.cc:377] Logged in from keytab as kudu/127.4.83.125@KRBTEST.COM (short username kudu)
I20260619 20:36:13.345369 27174 webserver.cc:460] Webserver started at http://127.4.83.125:33391/ using document root <none> and password file <none>
I20260619 20:36:13.345656 27174 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:13.345722 27174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:13.345964 27174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:13.347172 27167 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.83.125" port: 34851 } attempt: 2
I20260619 20:36:13.347266 27174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-1/data/instance:
uuid: "b202b995bf094c0693d9f6bf3186124f"
format_stamp: "Formatted at 2026-06-19 20:36:13 on dist-test-slave-4m58"
server_key: "7d10c704c863f30e0d91777c32d4320e"
server_key_iv: "db9a89d25e9f9bd1ddbbac25dbe9340d"
server_key_version: "encryptionkey@0"
I20260619 20:36:13.347569 27169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.347422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:34851 (local address 127.0.0.1:33508)
0619 20:36:13.347474 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:13.347507 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.125:34851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260619 20:36:13.347636 27174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-1/wal/instance:
uuid: "b202b995bf094c0693d9f6bf3186124f"
format_stamp: "Formatted at 2026-06-19 20:36:13 on dist-test-slave-4m58"
server_key: "7d10c704c863f30e0d91777c32d4320e"
server_key_iv: "db9a89d25e9f9bd1ddbbac25dbe9340d"
server_key_version: "encryptionkey@0"
I20260619 20:36:13.347942 27169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.347862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:34851 (local address 127.0.0.1:33524)
0619 20:36:13.347894 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:13.347922 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.125:34851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260619 20:36:13.348031 27167 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.83.125:34851: Network error: Client connection negotiation failed: client connection to 127.4.83.125:34851: connect: Connection refused (error 111)
I20260619 20:36:13.349157 27174 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:36:13.349893 27189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:13.350059 27174 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:36:13.350142 27174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-1/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-1/wal
uuid: "b202b995bf094c0693d9f6bf3186124f"
format_stamp: "Formatted at 2026-06-19 20:36:13 on dist-test-slave-4m58"
server_key: "7d10c704c863f30e0d91777c32d4320e"
server_key_iv: "db9a89d25e9f9bd1ddbbac25dbe9340d"
server_key_version: "encryptionkey@0"
I20260619 20:36:13.350211 27174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:13.359774 27174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:13.361847 27174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:13.361991 27174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:13.365844 27174 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.125:34851
I20260619 20:36:13.365880 27241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.125:34851 every 8 connection(s)
I20260619 20:36:13.366222 27174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-1/data/info.pb
I20260619 20:36:13.367156 27242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:13.368731 27242 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45623 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:36:13.373667 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 27174
I20260619 20:36:13.373780 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-1/wal/instance
I20260619 20:36:13.373981 4429 external_mini_cluster.cc:1428] Setting key 573aed2ee249d92427bb5d5618fe1824
Jun 19 20:36:13 dist-test-slave-4m58 krb5kdc[27086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901373, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.125@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:36:13.378033 27170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.369458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42242 (local address 127.4.83.126:45623)
0619 20:36:13.369548 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:13.369550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:13.369706 (+ 156us) server_negotiation.cc:407] Connection header received
0619 20:36:13.370199 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:13.370200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:13.370215 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:13.370267 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:13.370881 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.371358 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.372380 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.372579 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.372635 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:13.375022 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:13.375024 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:13.375024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:13.375031 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:13.376615 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:13.377040 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:13.377041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:13.377042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:13.377098 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:13.377330 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:13.377331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:13.377332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:13.377453 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:13.377509 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:13.377832 (+ 323us) server_negotiation.cc:299] Negotiation successful
0619 20:36:13.377927 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 20:36:13.378106 27244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.369371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:45623 (local address 127.0.0.1:42242)
0619 20:36:13.369585 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:13.369623 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:13.370100 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:13.370339 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:13.370342 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:13.370606 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:13.370793 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:13.370796 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.371450 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.371451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:13.372304 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:13.372307 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.372664 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.372666 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:13.372773 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:13.373268 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:13.373282 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:13.374903 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:13.376712 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:13.376719 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:13.376722 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:13.376953 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:13.377154 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:13.377155 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:13.377155 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:13.377266 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:13.377508 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:13.377515 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:13.377664 (+ 149us) client_negotiation.cc:769] Sending connection context
0619 20:36:13.377780 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 20:36:13.377927 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":94,"threads_started":1}
I20260619 20:36:13.378671 27242 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.125" port: 34851 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.4.83.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.83.124@KRBTEST.COM" created.
Jun 19 20:36:13 dist-test-slave-4m58 krb5kdc[27086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901373, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.125@KRBTEST.COM for kudu/127.4.83.125@KRBTEST.COM
I20260619 20:36:13.390549 27244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.378971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:34851 (local address 127.0.0.1:33540)
0619 20:36:13.379027 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:13.379039 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:13.379085 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:13.379497 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:13.379498 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:13.379669 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:13.379737 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:13.379739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.380764 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.380766 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:13.381552 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:13.381555 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.382017 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.382018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:13.382097 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:13.382529 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:13.382533 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:13.386691 (+ 4158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:13.389385 (+ 2694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:13.389387 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:13.389387 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:13.389573 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:13.389813 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:13.389814 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:13.389814 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:13.389879 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:13.390258 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:13.390259 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:13.390305 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 20:36:13.390364 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 20:36:13.390467 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 20:36:13.390714 27246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.379016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33540 (local address 127.4.83.125:34851)
0619 20:36:13.379314 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:13.379319 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:13.379342 (+ 23us) server_negotiation.cc:407] Connection header received
0619 20:36:13.379366 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:13.379368 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:13.379382 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:13.379463 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:13.379803 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.380683 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.381635 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.381946 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.382002 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:13.386812 (+ 4810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:13.386829 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:13.386829 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:13.386838 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:13.389255 (+ 2417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:13.389658 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:13.389662 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:13.389662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:13.389730 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:13.389951 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:13.389951 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:13.389951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:13.390179 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:13.390267 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:13.390416 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 20:36:13.390558 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":246,"thread_start_us":116,"threads_started":1}
I20260619 20:36:13.391336 27242 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.124" port: 46801 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:36:13.391996 27244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.391585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:46801 (local address 127.0.0.1:52642)
0619 20:36:13.391631 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:13.391678 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:46801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 20:36:13.392143 27193 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.83.124:46801: connect: Connection refused (error 111)
I20260619 20:36:13.392709 27244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.392591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:46801 (local address 127.0.0.1:52656)
0619 20:36:13.392655 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:13.392683 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:46801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260619 20:36:13.392817 27242 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.83.124:46801: Network error: Client connection negotiation failed: client connection to 127.4.83.124:46801: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.83.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.124.keytab.
Entry for principal kudu/127.4.83.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.124.keytab.
I20260619 20:36:13.403968 4429 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPZCFKi/build/release/bin/kudu
/tmp/dist-test-taskPZCFKi/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.124:46801
--webserver_interface=127.4.83.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.124.keytab
--principal=kudu/127.4.83.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.83.84:44447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.83.126:45623,127.4.83.125:34851,127.4.83.124:46801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:36:13.439203 27167 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.83.125" port: 34851 } attempt: 3
I20260619 20:36:13.442072 27242 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.83.124" port: 46801 } attempt: 1
I20260619 20:36:13.442507 27244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.442362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:46801 (local address 127.0.0.1:52658)
0619 20:36:13.442429 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:13.442472 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:46801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 20:36:13.442884 27244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.442799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:46801 (local address 127.0.0.1:52672)
0619 20:36:13.442840 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:13.442866 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:46801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 20:36:13.442971 27242 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.83.124:46801: Network error: Client connection negotiation failed: client connection to 127.4.83.124:46801: connect: Connection refused (error 111)
Jun 19 20:36:13 dist-test-slave-4m58 krb5kdc[27086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901373, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for kudu/127.4.83.125@KRBTEST.COM
I20260619 20:36:13.446224 27169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.439641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:34851 (local address 127.0.0.1:33552)
0619 20:36:13.439707 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:13.439720 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:13.439766 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:13.439932 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:13.439933 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:13.440122 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:13.440190 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:13.440192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.440859 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.440861 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:13.441390 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:13.441392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.441772 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.441773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:13.441843 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:13.442321 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:13.442326 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:13.443622 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:13.445280 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:13.445281 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:13.445282 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:13.445451 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:13.445667 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:13.445667 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:13.445667 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:13.445725 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:13.445970 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:13.445970 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:13.446019 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 20:36:13.446073 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 20:36:13.446155 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 20:36:13.446269 27246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.439729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33552 (local address 127.4.83.125:34851)
0619 20:36:13.439797 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:13.439800 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:13.439810 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:36:13.439858 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:13.439859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:13.439871 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:13.439918 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:13.440273 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.440779 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.441469 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.441706 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.441765 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:13.443702 (+ 1937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:13.443704 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:13.443704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:13.443712 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:13.445189 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:13.445552 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:13.445553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:13.445553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:13.445611 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:13.445771 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:13.445771 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:13.445772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:13.445907 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:13.445976 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:13.446106 (+ 130us) server_negotiation.cc:299] Negotiation successful
0619 20:36:13.446185 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 20:36:13.446645 27167 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.124" port: 46801 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:36:13.447013 27169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.446907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:46801 (local address 127.0.0.1:52684)
0619 20:36:13.446958 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:13.446989 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:46801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 20:36:13.447324 27169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.447247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:46801 (local address 127.0.0.1:52686)
0619 20:36:13.447284 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:13.447307 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:46801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260619 20:36:13.447414 27167 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.83.124:46801: Network error: Client connection negotiation failed: client connection to 127.4.83.124:46801: connect: Connection refused (error 111)
I20260619 20:36:13.478670 27167 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.83.124" port: 46801 } attempt: 1
I20260619 20:36:13.479411 27169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.479199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:46801 (local address 127.0.0.1:52692)
0619 20:36:13.479316 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:13.479366 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:46801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260619 20:36:13.479892 27169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.479727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:46801 (local address 127.0.0.1:52696)
0619 20:36:13.479791 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:13.479825 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:46801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260619 20:36:13.479993 27167 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.83.124:46801: Network error: Client connection negotiation failed: client connection to 127.4.83.124:46801: connect: Connection refused (error 111)
W20260619 20:36:13.487540 27249 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 20:36:13.487771 27249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:13.487787 27249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:13.487807 27249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:13.489004 27249 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:13.489053 27249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:13.489065 27249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:13.489077 27249 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:13.489087 27249 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:13.490613 27249 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.83.84:44447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.83.126:45623,127.4.83.125:34851,127.4.83.124:46801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/krb5kdc/kudu_127.4.83.124.keytab
--principal=kudu/127.4.83.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.83.124:46801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.83.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:13.490852 27249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:13.491077 27249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:13.495877 27254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:13.495966 27249 server_base.cc:1034] running on GCE node
W20260619 20:36:13.495877 27257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:13.495913 27255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:13.496384 27249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:13.496675 27249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:13.497828 27249 hybrid_clock.cc:648] HybridClock initialized: now 1781901373497811 us; error 32 us; skew 500 ppm
Jun 19 20:36:13 dist-test-slave-4m58 krb5kdc[27086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901373, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:13.500437 27249 init.cc:377] Logged in from keytab as kudu/127.4.83.124@KRBTEST.COM (short username kudu)
I20260619 20:36:13.500909 27249 webserver.cc:460] Webserver started at http://127.4.83.124:45211/ using document root <none> and password file <none>
I20260619 20:36:13.501135 27249 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:13.501183 27249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:13.501309 27249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:13.502363 27249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-2/data/instance:
uuid: "3e306b0d27f74df2aa2715ec6fd3b387"
format_stamp: "Formatted at 2026-06-19 20:36:13 on dist-test-slave-4m58"
server_key: "f5686b410f787cf49a9eba3959ec1164"
server_key_iv: "ca2cabc48e26305a5f50fbb46133f462"
server_key_version: "encryptionkey@0"
I20260619 20:36:13.502697 27249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-2/wal/instance:
uuid: "3e306b0d27f74df2aa2715ec6fd3b387"
format_stamp: "Formatted at 2026-06-19 20:36:13 on dist-test-slave-4m58"
server_key: "f5686b410f787cf49a9eba3959ec1164"
server_key_iv: "ca2cabc48e26305a5f50fbb46133f462"
server_key_version: "encryptionkey@0"
I20260619 20:36:13.504153 27249 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:36:13.505046 27264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:13.505252 27249 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:13.505342 27249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-2/data,/tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-2/wal
uuid: "3e306b0d27f74df2aa2715ec6fd3b387"
format_stamp: "Formatted at 2026-06-19 20:36:13 on dist-test-slave-4m58"
server_key: "f5686b410f787cf49a9eba3959ec1164"
server_key_iv: "ca2cabc48e26305a5f50fbb46133f462"
server_key_version: "encryptionkey@0"
I20260619 20:36:13.505406 27249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:13.511117 27242 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.83.124" port: 46801 } attempt: 2
I20260619 20:36:13.511633 27244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.511511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:46801 (local address 127.0.0.1:52708)
0619 20:36:13.511565 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:13.511604 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:46801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 20:36:13.512065 27244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.511942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:46801 (local address 127.0.0.1:52724)
0619 20:36:13.512011 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:13.512041 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.83.124:46801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260619 20:36:13.512174 27242 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.83.124:46801: Network error: Client connection negotiation failed: client connection to 127.4.83.124:46801: connect: Connection refused (error 111)
I20260619 20:36:13.531697 27249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:13.533834 27249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:13.533979 27249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:13.538079 27249 rpc_server.cc:307] RPC server started. Bound to: 127.4.83.124:46801
I20260619 20:36:13.538164 27316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.83.124:46801 every 8 connection(s)
I20260619 20:36:13.538473 27249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-2/data/info.pb
I20260619 20:36:13.538923 4429 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPZCFKi/build/release/bin/kudu as pid 27249
I20260619 20:36:13.539018 4429 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPZCFKi/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901102012824-4429-0/minicluster-data/master-2/wal/instance
I20260619 20:36:13.539199 4429 external_mini_cluster.cc:1428] Setting key df42416b255256deb0b4901373c63b4e
I20260619 20:36:13.539485 27317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:13.541571 27317 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45623 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 20:36:13 dist-test-slave-4m58 krb5kdc[27086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.124@KRBTEST.COM
Jun 19 20:36:13 dist-test-slave-4m58 krb5kdc[27086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901373, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.124@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:36:13.549867 27170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.542248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42256 (local address 127.4.83.126:45623)
0619 20:36:13.542307 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:13.542309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:13.542468 (+ 159us) server_negotiation.cc:407] Connection header received
0619 20:36:13.542540 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:13.542541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:13.542550 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:13.542594 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:13.543169 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.543896 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.544934 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.545141 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.545202 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:13.547426 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:13.547427 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:13.547428 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:13.547435 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:13.548816 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:13.549148 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:13.549149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:13.549149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:13.549201 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:13.549359 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:13.549360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:13.549360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:13.549469 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:13.549523 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:13.549688 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 20:36:13.549772 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 20:36:13.549970 27321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.542182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.126:45623 (local address 127.0.0.1:42256)
0619 20:36:13.542352 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:13.542396 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:13.542479 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:13.542629 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:13.542632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:13.542977 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:13.543072 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:13.543074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.544011 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.544012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:13.544831 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:13.544834 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.545242 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.545243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:13.545316 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:13.545919 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:13.545928 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:13.547341 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:13.548881 (+ 1540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:13.548888 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:13.548891 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:13.549083 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:13.549254 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:13.549255 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:13.549255 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:13.549312 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:13.549524 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:13.549532 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:13.549584 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 20:36:13.549657 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 20:36:13.549802 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":26,"thread_start_us":103,"threads_started":1}
I20260619 20:36:13.549973 27320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.539671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52728 (local address 127.4.83.124:46801)
0619 20:36:13.539858 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:13.539868 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:13.539901 (+ 33us) server_negotiation.cc:407] Connection header received
0619 20:36:13.540344 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:13.540350 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:13.540397 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:13.540526 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:13.541159 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.542169 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.543193 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.543910 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.544010 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:13.546612 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:13.546619 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:13.546620 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:13.546626 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:13.548393 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:13.548809 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:13.548812 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:13.548812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:13.548891 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:13.549083 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:13.549084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:13.549084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:13.549317 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:13.549431 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:13.549567 (+ 136us) server_negotiation.cc:299] Negotiation successful
0619 20:36:13.549741 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260619 20:36:13.550426 27317 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.125" port: 34851 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 20:36:13 dist-test-slave-4m58 krb5kdc[27086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901373, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.124@KRBTEST.COM for kudu/127.4.83.125@KRBTEST.COM
I20260619 20:36:13.555130 27167 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.83.124" port: 46801 } attempt: 2
Jun 19 20:36:13 dist-test-slave-4m58 krb5kdc[27086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.126@KRBTEST.COM
I20260619 20:36:13.557826 27246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.550919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33568 (local address 127.4.83.125:34851)
0619 20:36:13.550963 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:13.550965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:13.550972 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:36:13.550993 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:13.550994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:13.551004 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:13.551066 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:13.551379 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.552046 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.552973 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.553268 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.553358 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:13.555199 (+ 1841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:13.555200 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:13.555201 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:13.555208 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:13.556702 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:13.557069 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:13.557070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:13.557071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:13.557134 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:13.557327 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:13.557329 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:13.557329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:13.557466 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:13.557504 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:13.557668 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 20:36:13.557739 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:36:13.557826 27321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.550761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.125:34851 (local address 127.0.0.1:33568)
0619 20:36:13.550809 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:13.550819 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:13.550867 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:13.551074 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:13.551075 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:13.551236 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:13.551310 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:13.551312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.552136 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.552137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:13.552901 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:13.552903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.553358 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.553359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:13.553437 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:13.553854 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:13.553858 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:13.555119 (+ 1261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:13.556784 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:13.556785 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:13.556786 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:13.556996 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:13.557191 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:13.557191 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:13.557192 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:13.557248 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:13.557541 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:13.557542 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:13.557578 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 20:36:13.557630 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 20:36:13.557723 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 20:36:13.558169 27317 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.83.124" port: 46801 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 20:36:13 dist-test-slave-4m58 krb5kdc[27086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901373, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.126@KRBTEST.COM for kudu/127.4.83.124@KRBTEST.COM
I20260619 20:36:13.559598 27170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.550914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42270 (local address 127.4.83.126:45623)
0619 20:36:13.550961 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:13.550963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:13.550995 (+ 32us) server_negotiation.cc:407] Connection header received
0619 20:36:13.551091 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:13.551092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:13.551102 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:13.551150 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:13.551678 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.552328 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.553187 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.553437 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.553483 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:13.556302 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:13.556303 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:13.556303 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:13.556310 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:13.557809 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:13.558404 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:13.558405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:13.558406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:13.558628 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:13.558841 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:13.558842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:13.558842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:13.558967 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:13.559123 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:13.559412 (+ 289us) server_negotiation.cc:299] Negotiation successful
0619 20:36:13.559501 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
Jun 19 20:36:13 dist-test-slave-4m58 krb5kdc[27086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901373, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.124@KRBTEST.COM for kudu/127.4.83.124@KRBTEST.COM
I20260619 20:36:13.562992 27169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.555398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:46801 (local address 127.0.0.1:52736)
0619 20:36:13.555445 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:13.555456 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:13.555503 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:13.555638 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:13.555639 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:13.555789 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:13.555857 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:13.555859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.556610 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.556611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:13.557358 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:13.557360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.557830 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.557831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:13.557890 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:13.558348 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:13.558353 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:13.559621 (+ 1268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:13.561316 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:13.561317 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:13.561318 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:13.561521 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:13.561721 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:13.561722 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:13.561722 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:13.561786 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:13.562707 (+ 921us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:13.562708 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:13.562754 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 20:36:13.562799 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 20:36:13.562901 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260619 20:36:13.563328 27320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.555506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52736 (local address 127.4.83.124:46801)
0619 20:36:13.555547 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:13.555549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:13.555556 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:36:13.555576 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:13.555577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:13.555586 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:13.555636 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:13.555921 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.556541 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.557435 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.557739 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.559688 (+ 1949us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:13.559725 (+ 37us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:13.559725 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:13.559726 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:13.559729 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:13.561239 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:13.561597 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:13.561598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:13.561598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:13.561651 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:13.562489 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:13.562490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:13.562490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:13.562622 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:13.563113 (+ 491us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:13.563153 (+ 40us) server_negotiation.cc:299] Negotiation successful
0619 20:36:13.563243 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Jun 19 20:36:13 dist-test-slave-4m58 krb5kdc[27086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.83.125@KRBTEST.COM
I20260619 20:36:13.565272 27167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6e95c8bd7dc4d73b3ca73485d942950: Bootstrap starting.
I20260619 20:36:13.565449 27321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.558395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:46801 (local address 127.0.0.1:52744)
0619 20:36:13.558466 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:13.558475 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:13.558509 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:13.558737 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:13.558738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:13.558856 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:13.558882 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:13.558883 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.560148 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.560148 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:13.560751 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:13.560753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.561089 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.561090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:13.561134 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:13.561444 (+ 310us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:13.561449 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:13.562674 (+ 1225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:13.564398 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:13.564399 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:13.564400 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:13.564584 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:13.564748 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:13.564749 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:13.564749 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:13.564800 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:13.565075 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:13.565076 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:13.565113 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 20:36:13.565215 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 20:36:13.565313 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260619 20:36:13.565668 27322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.558476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52744 (local address 127.4.83.124:46801)
0619 20:36:13.558622 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:13.558623 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:13.558629 (+ 6us) server_negotiation.cc:407] Connection header received
0619 20:36:13.558644 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:13.558645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:13.558654 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:13.558730 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:13.558940 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.560083 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.560818 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.561034 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.561088 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:13.562745 (+ 1657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:13.562746 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:13.562746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:13.562752 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:13.564316 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:13.564645 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:13.564646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:13.564647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:13.564698 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:13.564847 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:13.564848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:13.564848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:13.564984 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:13.565037 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:13.565244 (+ 207us) server_negotiation.cc:299] Negotiation successful
0619 20:36:13.565382 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":129,"thread_start_us":102,"threads_started":1}
I20260619 20:36:13.566054 27167 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6e95c8bd7dc4d73b3ca73485d942950: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:13.566359 27167 log.cc:824] T 00000000000000000000000000000000 P b6e95c8bd7dc4d73b3ca73485d942950: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:13.567029 27317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e306b0d27f74df2aa2715ec6fd3b387: Bootstrap starting.
I20260619 20:36:13.567175 27167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6e95c8bd7dc4d73b3ca73485d942950: No bootstrap required, opened a new log
I20260619 20:36:13.567703 27317 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e306b0d27f74df2aa2715ec6fd3b387: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:13.568051 27317 log.cc:824] T 00000000000000000000000000000000 P 3e306b0d27f74df2aa2715ec6fd3b387: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:13.568220 27246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.560487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33578 (local address 127.4.83.125:34851)
0619 20:36:13.560525 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:13.560527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:13.560550 (+ 23us) server_negotiation.cc:407] Connection header received
0619 20:36:13.560568 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:13.560569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:13.560577 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:13.560618 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:13.561173 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.561809 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.562700 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.562927 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.562981 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:13.565492 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:13.565493 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:13.565494 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:13.565501 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:13.566935 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:13.567428 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:13.567429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:13.567430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:13.567489 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:13.567689 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:13.567690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:13.567690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:13.567816 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:13.568018 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:13.568058 (+ 40us) server_negotiation.cc:299] Negotiation successful
0619 20:36:13.568137 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:36:13.568931 27317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e306b0d27f74df2aa2715ec6fd3b387: No bootstrap required, opened a new log
I20260619 20:36:13.569196 27167 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6e95c8bd7dc4d73b3ca73485d942950 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e95c8bd7dc4d73b3ca73485d942950" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45623 } } peers { permanent_uuid: "b202b995bf094c0693d9f6bf3186124f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 34851 } } peers { permanent_uuid: "3e306b0d27f74df2aa2715ec6fd3b387" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 46801 } }
I20260619 20:36:13.569332 27167 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6e95c8bd7dc4d73b3ca73485d942950 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6e95c8bd7dc4d73b3ca73485d942950, State: Initialized, Role: FOLLOWER
I20260619 20:36:13.569418 27167 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6e95c8bd7dc4d73b3ca73485d942950 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e95c8bd7dc4d73b3ca73485d942950" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45623 } } peers { permanent_uuid: "b202b995bf094c0693d9f6bf3186124f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 34851 } } peers { permanent_uuid: "3e306b0d27f74df2aa2715ec6fd3b387" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 46801 } }
I20260619 20:36:13.569748 27167 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6e95c8bd7dc4d73b3ca73485d942950 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:13.569675 27325 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e95c8bd7dc4d73b3ca73485d942950 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e95c8bd7dc4d73b3ca73485d942950" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45623 } } peers { permanent_uuid: "b202b995bf094c0693d9f6bf3186124f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 34851 } } peers { permanent_uuid: "3e306b0d27f74df2aa2715ec6fd3b387" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 46801 } } }
I20260619 20:36:13.570003 4429 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:36:13.569998 27325 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e95c8bd7dc4d73b3ca73485d942950 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:36:13.571489 27317 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e306b0d27f74df2aa2715ec6fd3b387 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e95c8bd7dc4d73b3ca73485d942950" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45623 } } peers { permanent_uuid: "b202b995bf094c0693d9f6bf3186124f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 34851 } } peers { permanent_uuid: "3e306b0d27f74df2aa2715ec6fd3b387" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 46801 } }
I20260619 20:36:13.571668 27317 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e306b0d27f74df2aa2715ec6fd3b387 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e306b0d27f74df2aa2715ec6fd3b387, State: Initialized, Role: FOLLOWER
I20260619 20:36:13.571703 27141 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b6e95c8bd7dc4d73b3ca73485d942950"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42270
I20260619 20:36:13.571760 27317 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e306b0d27f74df2aa2715ec6fd3b387 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e95c8bd7dc4d73b3ca73485d942950" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45623 } } peers { permanent_uuid: "b202b995bf094c0693d9f6bf3186124f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 34851 } } peers { permanent_uuid: "3e306b0d27f74df2aa2715ec6fd3b387" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 46801 } }
I20260619 20:36:13.571859 27141 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6e95c8bd7dc4d73b3ca73485d942950 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 20:36:13.571909 27141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6e95c8bd7dc4d73b3ca73485d942950 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:13.572122 27317 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e306b0d27f74df2aa2715ec6fd3b387 [sys.catalog]: configured and running, proceeding with master startup.
W20260619 20:36:13.572525 27336 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b6e95c8bd7dc4d73b3ca73485d942950: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:36:13.572640 27336 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:36:13.572816 27141 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6e95c8bd7dc4d73b3ca73485d942950 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e95c8bd7dc4d73b3ca73485d942950" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45623 } } peers { permanent_uuid: "b202b995bf094c0693d9f6bf3186124f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 34851 } } peers { permanent_uuid: "3e306b0d27f74df2aa2715ec6fd3b387" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 46801 } }
I20260619 20:36:13.573055 27141 leader_election.cc:288] T 00000000000000000000000000000000 P b6e95c8bd7dc4d73b3ca73485d942950 [CANDIDATE]: Term 1 election: Requested vote from peers b202b995bf094c0693d9f6bf3186124f (127.4.83.125:34851), 3e306b0d27f74df2aa2715ec6fd3b387 (127.4.83.124:46801)
W20260619 20:36:13.574122 27349 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3e306b0d27f74df2aa2715ec6fd3b387: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:36:13.574609 27349 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:36:13.574107 27291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6e95c8bd7dc4d73b3ca73485d942950" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3e306b0d27f74df2aa2715ec6fd3b387"
I20260619 20:36:13.574783 27291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e306b0d27f74df2aa2715ec6fd3b387 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:13.575758 27291 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3e306b0d27f74df2aa2715ec6fd3b387 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6e95c8bd7dc4d73b3ca73485d942950 in term 1.
I20260619 20:36:13.576013 27118 leader_election.cc:302] T 00000000000000000000000000000000 P b6e95c8bd7dc4d73b3ca73485d942950 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e306b0d27f74df2aa2715ec6fd3b387, b6e95c8bd7dc4d73b3ca73485d942950; no voters:
I20260619 20:36:13.576143 27325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6e95c8bd7dc4d73b3ca73485d942950 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:13.576297 27325 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6e95c8bd7dc4d73b3ca73485d942950 [term 1 LEADER]: Becoming Leader. State: Replica: b6e95c8bd7dc4d73b3ca73485d942950, State: Running, Role: LEADER
I20260619 20:36:13.576390 27325 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6e95c8bd7dc4d73b3ca73485d942950 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e95c8bd7dc4d73b3ca73485d942950" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45623 } } peers { permanent_uuid: "b202b995bf094c0693d9f6bf3186124f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 34851 } } peers { permanent_uuid: "3e306b0d27f74df2aa2715ec6fd3b387" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 46801 } }
I20260619 20:36:13.575075 27099 master_runner.cc:186] Error getting master registration for 127.4.83.125:34851: OK, instance_id { permanent_uuid: "b202b995bf094c0693d9f6bf3186124f" instance_seqno: 1781901373364762 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 20:36:13.577492 27099 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 20:36:13.578907 27351 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e95c8bd7dc4d73b3ca73485d942950 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6e95c8bd7dc4d73b3ca73485d942950. Latest consensus state: current_term: 1 leader_uuid: "b6e95c8bd7dc4d73b3ca73485d942950" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e95c8bd7dc4d73b3ca73485d942950" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45623 } } peers { permanent_uuid: "b202b995bf094c0693d9f6bf3186124f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 34851 } } peers { permanent_uuid: "3e306b0d27f74df2aa2715ec6fd3b387" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 46801 } } }
I20260619 20:36:13.579082 27351 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e95c8bd7dc4d73b3ca73485d942950 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:13.579555 27337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e306b0d27f74df2aa2715ec6fd3b387 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e95c8bd7dc4d73b3ca73485d942950" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45623 } } peers { permanent_uuid: "b202b995bf094c0693d9f6bf3186124f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 34851 } } peers { permanent_uuid: "3e306b0d27f74df2aa2715ec6fd3b387" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 46801 } } }
I20260619 20:36:13.579746 27337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e306b0d27f74df2aa2715ec6fd3b387 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:36:13.578240 27249 master_runner.cc:186] Error getting master registration for 127.4.83.125:34851: OK, instance_id { permanent_uuid: "b202b995bf094c0693d9f6bf3186124f" instance_seqno: 1781901373364762 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 20:36:13.581038 27249 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 20:36:13.584851 27357 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:36:13.585330 27357 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:36:13.593713 27291 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3e306b0d27f74df2aa2715ec6fd3b387 [term 1 FOLLOWER]: Refusing update from remote peer b6e95c8bd7dc4d73b3ca73485d942950: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:13.594148 27325 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b6e95c8bd7dc4d73b3ca73485d942950 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e306b0d27f74df2aa2715ec6fd3b387" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 46801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:13.597000 27170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.584413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42276 (local address 127.4.83.126:45623)
0619 20:36:13.584470 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:13.584472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:13.584840 (+ 368us) server_negotiation.cc:407] Connection header received
0619 20:36:13.584999 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:13.585001 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:13.585013 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:13.585046 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:13.586462 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.587196 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.589032 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.589352 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.589393 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:13.591299 (+ 1906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:13.591301 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:13.591301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:13.591310 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:13.593001 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:13.595782 (+ 2781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:13.595783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:13.595784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:13.595899 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:13.596134 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:13.596135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:13.596135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:13.596283 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:13.596391 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:13.596818 (+ 427us) server_negotiation.cc:299] Negotiation successful
0619 20:36:13.596901 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 20:36:13.597393 27337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e306b0d27f74df2aa2715ec6fd3b387 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6e95c8bd7dc4d73b3ca73485d942950. Latest consensus state: current_term: 1 leader_uuid: "b6e95c8bd7dc4d73b3ca73485d942950" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e95c8bd7dc4d73b3ca73485d942950" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45623 } } peers { permanent_uuid: "b202b995bf094c0693d9f6bf3186124f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 34851 } } peers { permanent_uuid: "3e306b0d27f74df2aa2715ec6fd3b387" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 46801 } } }
I20260619 20:36:13.597517 27337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e306b0d27f74df2aa2715ec6fd3b387 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:36:13.600127 27246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.584667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33584 (local address 127.4.83.125:34851)
0619 20:36:13.584713 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:13.584715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:13.592006 (+ 7291us) server_negotiation.cc:407] Connection header received
0619 20:36:13.592113 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:13.592114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:13.592128 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:13.592158 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:13.592786 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.593468 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.594358 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.594640 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.594681 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:13.596513 (+ 1832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:13.596515 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:13.596515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:13.596534 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:13.598281 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:13.598883 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:13.598885 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:13.598885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:13.598974 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:13.599271 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:13.599272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:13.599272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:13.599429 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:13.599503 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:13.599897 (+ 394us) server_negotiation.cc:299] Negotiation successful
0619 20:36:13.600014 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:36:13.603300 27242 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.83.124" port: 46801 } attempt: 3
I20260619 20:36:13.604779 27325 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e95c8bd7dc4d73b3ca73485d942950 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b6e95c8bd7dc4d73b3ca73485d942950" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e95c8bd7dc4d73b3ca73485d942950" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45623 } } peers { permanent_uuid: "b202b995bf094c0693d9f6bf3186124f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 34851 } } peers { permanent_uuid: "3e306b0d27f74df2aa2715ec6fd3b387" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 46801 } } }
I20260619 20:36:13.604907 27325 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e95c8bd7dc4d73b3ca73485d942950 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:13.604984 27361 mvcc.cc:205] Tried to move back new op lower bound from 7298668026235588608 to 7298668026169659392. Current Snapshot: MvccSnapshot[applied={T|T < 7298668026235588608}]
I20260619 20:36:13.606374 27337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e306b0d27f74df2aa2715ec6fd3b387 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b6e95c8bd7dc4d73b3ca73485d942950" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e95c8bd7dc4d73b3ca73485d942950" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45623 } } peers { permanent_uuid: "b202b995bf094c0693d9f6bf3186124f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 34851 } } peers { permanent_uuid: "3e306b0d27f74df2aa2715ec6fd3b387" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 46801 } } }
I20260619 20:36:13.606490 27337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e306b0d27f74df2aa2715ec6fd3b387 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:36:13.607399 27357 catalog_manager.cc:1347] Generated new cluster ID: 99c7fa978b0044738b95b9f86c83334b
I20260619 20:36:13.607455 27357 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:36:13 dist-test-slave-4m58 krb5kdc[27086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901373, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.83.125@KRBTEST.COM for kudu/127.4.83.124@KRBTEST.COM
I20260619 20:36:13.613654 27244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.603576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.83.124:46801 (local address 127.0.0.1:52766)
0619 20:36:13.603663 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:13.603674 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:13.603712 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:13.604008 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:13.604009 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:13.604183 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:13.604233 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:13.604235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.605210 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.605211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:13.605948 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:13.605951 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.607223 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.607225 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:13.607286 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:13.607713 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:13.607718 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:13.610089 (+ 2371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:13.612397 (+ 2308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:13.612398 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:13.612399 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:13.612625 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:13.612955 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:13.612956 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:13.612956 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:13.613047 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:13.613325 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:13.613326 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:13.613373 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 20:36:13.613428 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 20:36:13.613544 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260619 20:36:13.614203 27320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.603775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52766 (local address 127.4.83.124:46801)
0619 20:36:13.603812 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:13.603814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:13.603820 (+ 6us) server_negotiation.cc:407] Connection header received
0619 20:36:13.603868 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:13.603869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:13.603878 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:13.603923 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:13.604317 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.605002 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.606807 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.607098 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.607163 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:13.610204 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:13.610206 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:13.610206 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:13.610214 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:13.612294 (+ 2080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:13.612804 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:13.612806 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:13.612806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:13.612875 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:13.613107 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:13.613108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:13.613108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:13.613250 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:13.613302 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:13.614019 (+ 717us) server_negotiation.cc:299] Negotiation successful
0619 20:36:13.614112 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:36:13.615345 27322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:13.584236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52760 (local address 127.4.83.124:46801)
0619 20:36:13.584291 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:13.584294 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:13.592198 (+ 7904us) server_negotiation.cc:407] Connection header received
0619 20:36:13.592232 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:13.592233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:13.592246 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:13.592294 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:13.598180 (+ 5886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.598935 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.599962 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:13.600288 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:13.600335 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:13.602981 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:13.602983 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:13.602983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:13.602993 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:13.606620 (+ 3627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:13.611243 (+ 4623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:13.611245 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:13.611245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:13.611305 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:13.614674 (+ 3369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:13.614675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:13.614675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:13.614807 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:13.614914 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:13.615180 (+ 266us) server_negotiation.cc:299] Negotiation successful
0619 20:36:13.615263 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 20:36:13.615928 27362 mvcc.cc:205] Tried to move back new op lower bound from 7298668026235588608 to 7298668026169659392. Current Snapshot: MvccSnapshot[applied={T|T < 7298668026235588608}]
I20260619 20:36:13.616015 27242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b202b995bf094c0693d9f6bf3186124f: Bootstrap starting.
I20260619 20:36:13.616766 27242 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b202b995bf094c0693d9f6bf3186124f: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:13.617057 27242 log.cc:824] T 00000000000000000000000000000000 P b202b995bf094c0693d9f6bf3186124f: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:13.617995 27242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b202b995bf094c0693d9f6bf3186124f: No bootstrap required, opened a new log
I20260619 20:36:13.619593 27242 raft_consensus.cc:348] T 00000000000000000000000000000000 P b202b995bf094c0693d9f6bf3186124f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e95c8bd7dc4d73b3ca73485d942950" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45623 } } peers { permanent_uuid: "b202b995bf094c0693d9f6bf3186124f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 34851 } } peers { permanent_uuid: "3e306b0d27f74df2aa2715ec6fd3b387" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 46801 } }
I20260619 20:36:13.619720 27242 raft_consensus.cc:729] T 00000000000000000000000000000000 P b202b995bf094c0693d9f6bf3186124f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b202b995bf094c0693d9f6bf3186124f, State: Initialized, Role: FOLLOWER
I20260619 20:36:13.619796 27242 consensus_queue.cc:260] T 00000000000000000000000000000000 P b202b995bf094c0693d9f6bf3186124f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e95c8bd7dc4d73b3ca73485d942950" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45623 } } peers { permanent_uuid: "b202b995bf094c0693d9f6bf3186124f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 34851 } } peers { permanent_uuid: "3e306b0d27f74df2aa2715ec6fd3b387" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 46801 } }
I20260619 20:36:13.620849 27242 sys_catalog.cc:564] T 00000000000000000000000000000000 P b202b995bf094c0693d9f6bf3186124f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:13.620929 27215 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b202b995bf094c0693d9f6bf3186124f [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:13.622551 27215 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b202b995bf094c0693d9f6bf3186124f [term 1 FOLLOWER]: Refusing update from remote peer b6e95c8bd7dc4d73b3ca73485d942950: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:13.622901 27351 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b6e95c8bd7dc4d73b3ca73485d942950 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b202b995bf094c0693d9f6bf3186124f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 34851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:13.623476 27216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6e95c8bd7dc4d73b3ca73485d942950" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b202b995bf094c0693d9f6bf3186124f"
I20260619 20:36:13.624469 27216 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b202b995bf094c0693d9f6bf3186124f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6e95c8bd7dc4d73b3ca73485d942950 in term 1.
I20260619 20:36:13.626307 27381 sys_catalog.cc:455] T 00000000000000000000000000000000 P b202b995bf094c0693d9f6bf3186124f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6e95c8bd7dc4d73b3ca73485d942950" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e95c8bd7dc4d73b3ca73485d942950" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45623 } } peers { permanent_uuid: "b202b995bf094c0693d9f6bf3186124f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 34851 } } peers { permanent_uuid: "3e306b0d27f74df2aa2715ec6fd3b387" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 46801 } } }
I20260619 20:36:13.626628 27381 sys_catalog.cc:458] T 00000000000000000000000000000000 P b202b995bf094c0693d9f6bf3186124f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:36:13.626744 27381 sys_catalog.cc:455] T 00000000000000000000000000000000 P b202b995bf094c0693d9f6bf3186124f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6e95c8bd7dc4d73b3ca73485d942950. Latest consensus state: current_term: 1 leader_uuid: "b6e95c8bd7dc4d73b3ca73485d942950" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e95c8bd7dc4d73b3ca73485d942950" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45623 } } peers { permanent_uuid: "b202b995bf094c0693d9f6bf3186124f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 34851 } } peers { permanent_uuid: "3e306b0d27f74df2aa2715ec6fd3b387" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 46801 } } }
I20260619 20:36:13.626837 27381 sys_catalog.cc:458] T 00000000000000000000000000000000 P b202b995bf094c0693d9f6bf3186124f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:36:13.626895 27381 sys_catalog.cc:455] T 00000000000000000000000000000000 P b202b995bf094c0693d9f6bf3186124f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b6e95c8bd7dc4d73b3ca73485d942950" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e95c8bd7dc4d73b3ca73485d942950" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45623 } } peers { permanent_uuid: "b202b995bf094c0693d9f6bf3186124f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 34851 } } peers { permanent_uuid: "3e306b0d27f74df2aa2715ec6fd3b387" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 46801 } } }
I20260619 20:36:13.626968 27381 sys_catalog.cc:458] T 00000000000000000000000000000000 P b202b995bf094c0693d9f6bf3186124f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:36:13.628211 27379 mvcc.cc:205] Tried to move back new op lower bound from 7298668026235588608 to 7298668026169659392. Current Snapshot: MvccSnapshot[applied={T|T < 7298668026235588608}]
I20260619 20:36:13.632539 27382 catalog_manager.cc:1259] Loaded cluster ID: 99c7fa978b0044738b95b9f86c83334b
I20260619 20:36:13.632678 27382 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b202b995bf094c0693d9f6bf3186124f: loading cluster ID for follower catalog manager: success
W20260619 20:36:13.632803 27382 catalog_manager.cc:1576] T 00000000000000000000000000000000 P b202b995bf094c0693d9f6bf3186124f: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260619 20:36:13.632875 27382 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:36:13.648803 27351 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e95c8bd7dc4d73b3ca73485d942950 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b6e95c8bd7dc4d73b3ca73485d942950" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e95c8bd7dc4d73b3ca73485d942950" member_type: VOTER last_known_addr { host: "127.4.83.126" port: 45623 } } peers { permanent_uuid: "b202b995bf094c0693d9f6bf3186124f" member_type: VOTER last_known_addr { host: "127.4.83.125" port: 34851 } } peers { permanent_uuid: "3e306b0d27f74df2aa2715ec6fd3b387" member_type: VOTER last_known_addr { host: "127.4.83.124" port: 46801 } } }
I20260619 20:36:13.648933 27351 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e95c8bd7dc4d73b3ca73485d942950 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:13.651656 27357 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:36:13.652388 27357 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:36:13.658298 27357 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6e95c8bd7dc4d73b3ca73485d942950: Generated new TSK 0
I20260619 20:36:13.658665 27357 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 20:36:13.687342 27322 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:13.685614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52780 (local address 127.4.83.124:46801)
0619 20:36:13.685665 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:13.685667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:13.686623 (+ 956us) server_negotiation.cc:407] Connection header received
0619 20:36:13.686730 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:13.686732 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:13.686743 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:13.686772 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:13.687259 (+ 487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52780: BlockingRecv error: recv got EOF from 127.0.0.1:52780 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260619 20:36:13.688356 27170 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:13.686056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42292 (local address 127.4.83.126:45623)
0619 20:36:13.686095 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:13.686097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:13.687773 (+ 1676us) server_negotiation.cc:407] Connection header received
0619 20:36:13.687903 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:13.687904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:13.687915 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:13.687944 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:13.688286 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42292: BlockingRecv error: recv got EOF from 127.0.0.1:42292 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260619 20:36:13.689230 27246 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:13.686366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33600 (local address 127.4.83.125:34851)
0619 20:36:13.686404 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:13.686406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:13.688682 (+ 2276us) server_negotiation.cc:407] Connection header received
0619 20:36:13.688813 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:13.688814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:13.688825 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:13.688854 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:13.689169 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33600: BlockingRecv error: recv got EOF from 127.0.0.1:33600 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 20:36:13.691773 27246 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:13.690992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33602 (local address 127.4.83.125:34851)
0619 20:36:13.691031 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:13.691033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:13.691229 (+ 196us) server_negotiation.cc:407] Connection header received
0619 20:36:13.691329 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:13.691330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:13.691349 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:13.691377 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:13.691721 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33602: BlockingRecv error: recv got EOF from 127.0.0.1:33602 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260619 20:36:13.695013 27246 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:13.694222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33612 (local address 127.4.83.125:34851)
0619 20:36:13.694257 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:13.694259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:13.694494 (+ 235us) server_negotiation.cc:407] Connection header received
0619 20:36:13.694587 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:13.694588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:13.694598 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:13.694628 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:13.694945 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33612: BlockingRecv error: recv got EOF from 127.0.0.1:33612 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 20:36:13.699240 27246 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:13.698345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33624 (local address 127.4.83.125:34851)
0619 20:36:13.698382 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:13.698384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:13.698650 (+ 266us) server_negotiation.cc:407] Connection header received
0619 20:36:13.698771 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:13.698772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:13.698784 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:13.698813 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:13.699183 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33624: BlockingRecv error: recv got EOF from 127.0.0.1:33624 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 20:36:13.705711 27246 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:13.704833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33640 (local address 127.4.83.125:34851)
0619 20:36:13.704875 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:13.704877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:13.705130 (+ 253us) server_negotiation.cc:407] Connection header received
0619 20:36:13.705227 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:13.705228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:13.705240 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:13.705268 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:13.705649 (+ 381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33640: BlockingRecv error: recv got EOF from 127.0.0.1:33640 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 20:36:13.716280 27246 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:13.715294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33646 (local address 127.4.83.125:34851)
0619 20:36:13.715353 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:13.715355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:13.715648 (+ 293us) server_negotiation.cc:407] Connection header received
0619 20:36:13.715744 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:13.715745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:13.715757 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:13.715784 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:13.716215 (+ 431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33646: BlockingRecv error: recv got EOF from 127.0.0.1:33646 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260619 20:36:13.734903 27246 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:13.733923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33654 (local address 127.4.83.125:34851)
0619 20:36:13.733967 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:13.733969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:13.734257 (+ 288us) server_negotiation.cc:407] Connection header received
0619 20:36:13.734364 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:13.734365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:13.734377 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:13.734422 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:13.734838 (+ 416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33654: BlockingRecv error: recv got EOF from 127.0.0.1:33654 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 20:36:13.769493 27246 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:13.768567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33658 (local address 127.4.83.125:34851)
0619 20:36:13.768610 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:13.768612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:13.768871 (+ 259us) server_negotiation.cc:407] Connection header received
0619 20:36:13.768970 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:13.768971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:13.768984 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:13.769012 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:13.769427 (+ 415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33658: BlockingRecv error: recv got EOF from 127.0.0.1:33658 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260619 20:36:13.835891 27246 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:13.835163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33674 (local address 127.4.83.125:34851)
0619 20:36:13.835229 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:13.835231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:13.835287 (+ 56us) server_negotiation.cc:407] Connection header received
0619 20:36:13.835338 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:13.835339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:13.835349 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:13.835416 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:13.835792 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33674: BlockingRecv error: recv got EOF from 127.0.0.1:33674 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260619 20:36:13.965569 27246 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:13.965039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33688 (local address 127.4.83.125:34851)
0619 20:36:13.965113 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:13.965115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:13.965127 (+ 12us) server_negotiation.cc:407] Connection header received
0619 20:36:13.965154 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:13.965155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:13.965164 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:13.965233 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:13.965521 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33688: BlockingRecv error: recv got EOF from 127.0.0.1:33688 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260619 20:36:14.223664 27246 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:14.222952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33704 (local address 127.4.83.125:34851)
0619 20:36:14.223032 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:14.223034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:14.223069 (+ 35us) server_negotiation.cc:407] Connection header received
0619 20:36:14.223095 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:14.223096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:14.223106 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:14.223148 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:14.223596 (+ 448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33704: BlockingRecv error: recv got EOF from 127.0.0.1:33704 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 20:36:14.575222 27349 catalog_manager.cc:1259] Loaded cluster ID: 99c7fa978b0044738b95b9f86c83334b
I20260619 20:36:14.575321 27349 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3e306b0d27f74df2aa2715ec6fd3b387: loading cluster ID for follower catalog manager: success
I20260619 20:36:14.576277 27349 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3e306b0d27f74df2aa2715ec6fd3b387: acquiring CA information for follower catalog manager: success
I20260619 20:36:14.576622 27349 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3e306b0d27f74df2aa2715ec6fd3b387: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 20:36:14.634286 27382 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b202b995bf094c0693d9f6bf3186124f: acquiring CA information for follower catalog manager: success
I20260619 20:36:14.634719 27382 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b202b995bf094c0693d9f6bf3186124f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 20:36:14.740013 27449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:14.737096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33710 (local address 127.4.83.125:34851)
0619 20:36:14.737262 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:14.737264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:14.737276 (+ 12us) server_negotiation.cc:407] Connection header received
0619 20:36:14.737311 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:14.737312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:14.737395 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:14.737477 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:14.737649 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:14.738237 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:14.739096 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:14.739326 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:14.739387 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:14.739567 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:14.739720 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:14.739806 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:14.739866 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 20:36:14.739897 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":66,"threads_started":1}
I20260619 20:36:14.744984 27455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:14.742060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52792 (local address 127.4.83.124:46801)
0619 20:36:14.742217 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:14.742219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:14.742230 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:36:14.742393 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:14.742394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:14.742449 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:14.742543 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:14.742655 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:14.743206 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:14.744124 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:14.744353 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:14.744411 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:14.744581 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:14.744730 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:14.744794 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:14.744869 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 20:36:14.744894 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":71,"threads_started":1}
I20260619 20:36:14.749778 27449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:14.747129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33720 (local address 127.4.83.125:34851)
0619 20:36:14.747179 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:14.747181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:14.747188 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:36:14.747253 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:14.747253 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:14.747264 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:14.747318 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:14.747428 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:14.748058 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:14.748936 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:14.749240 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:14.749302 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:14.749477 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:14.749517 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:14.749559 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:14.749641 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 20:36:14.749673 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 20:36:14.749867 27455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:14.746863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52794 (local address 127.4.83.124:46801)
0619 20:36:14.746914 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:14.746916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:14.747017 (+ 101us) server_negotiation.cc:407] Connection header received
0619 20:36:14.747107 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:14.747108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:14.747120 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:14.747177 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:14.747297 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:14.748016 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:14.748978 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:14.749270 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:14.749326 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:14.749591 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:14.749626 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:14.749672 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:14.749756 (+ 84us) server_negotiation.cc:299] Negotiation successful
0619 20:36:14.749784 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:36:14.750193 27462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:14.746922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42296 (local address 127.4.83.126:45623)
0619 20:36:14.747232 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:14.747235 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:14.747242 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:36:14.747300 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:14.747301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:14.747371 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:14.747470 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:14.747668 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:14.748445 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:14.749275 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:14.749553 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:14.749608 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:14.749790 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:14.749926 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:14.749985 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:14.750079 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 20:36:14.750108 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":73,"threads_started":1}
I20260619 20:36:14.757134 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 27099
I20260619 20:36:14.762589 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 27174
I20260619 20:36:14.767659 4429 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPZCFKi/build/release/bin/kudu with pid 27249
2026-06-19T20:36:14Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2774 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5557 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (272763 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST